Publications

2017

  • A. Abdolmaleki, B. Price, N. Lau, P. Reis, and G. Neumann, “Contextual CMA-ES,” in International Joint Conference on Artificial Intelligence (IJCAI), 2017.
    [BibTeX] [Abstract] [EPrints]

    Many stochastic search algorithms are designed to optimize a fixed objective function to learn a task, i.e., if the objective function changes slightly, for example, due to a change in the situation or context of the task, relearning is required to adapt to the new context. For instance, if we want to learn a kicking movement for a soccer robot, we have to relearn the movement for different ball locations. Such relearning is undesired as it is highly inefficient and many applications require a fast adaptation to a new context/situation. Therefore, we investigate contextual stochastic search algorithms that can learn multiple, similar tasks simultaneously. Current contextual stochastic search methods are based on policy search algorithms and suffer from premature convergence and the need for parameter tuning. In this paper, we extend the well known CMA-ES algorithm to the contextual setting and illustrate its performance on several contextual tasks. Our new algorithm, called contextual CMAES, leverages from contextual learning while it preserves all the features of standard CMA-ES such as stability, avoidance of premature convergence, step size control and a minimal amount of parameter tuning.

    @inproceedings{lirolem28141,
           booktitle = {International Joint Conference on Artificial Intelligence (IJCAI)},
               month = {August},
               title = {Contextual CMA-ES},
              author = {A. Abdolmaleki and B. Price and N. Lau and P. Reis and G. Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592ee6d0)},
                 url = {http://eprints.lincoln.ac.uk/28141/},
            abstract = {Many stochastic search algorithms are designed to optimize a fixed objective function to learn a task, i.e., if the objective function changes slightly, for example, due to a change in the situation or context of the task, relearning is required to adapt to the new context. For instance, if we want to learn a kicking movement for a soccer robot, we have to relearn the movement for different ball locations. Such relearning is undesired as it is highly inefficient and many applications require a fast adaptation to a new context/situation. Therefore, we investigate contextual stochastic search algorithms
    that can learn multiple, similar tasks simultaneously. Current contextual stochastic search methods are based on policy search algorithms and suffer from premature convergence and the need for parameter tuning. In this paper, we extend the well known CMA-ES algorithm to the contextual setting and illustrate its performance on several contextual
    tasks. Our new algorithm, called contextual CMAES, leverages from contextual learning while it preserves all the features of standard CMA-ES such as stability, avoidance of premature convergence, step size control and a minimal amount of parameter tuning.}
    }
  • A. Abdolmaleki, B. Price, N. Lau, L. P. Reis, and G. Neumann, “Deriving and improving CMA-ES with Information geometric trust regions,” in The Genetic and Evolutionary Computation Conference (GECCO 2017), 2017.
    [BibTeX] [Abstract] [EPrints]

    CMA-ES is one of the most popular stochastic search algorithms. It performs favourably in many tasks without the need of extensive parameter tuning. The algorithm has many beneficial properties, including automatic step-size adaptation, efficient covariance updates that incorporates the current samples as well as the evolution path and its invariance properties. Its update rules are composed of well established heuristics where the theoretical foundations of some of these rules are also well understood. In this paper we will fully derive all CMA-ES update rules within the framework of expectation-maximisation-based stochastic search algorithms using information-geometric trust regions. We show that the use of the trust region results in similar updates to CMA-ES for the mean and the covariance matrix while it allows for the derivation of an improved update rule for the step-size. Our new algorithm, Trust-Region Covariance Matrix Adaptation Evolution Strategy (TR-CMA-ES) is fully derived from first order optimization principles and performs favourably in compare to standard CMA-ES algorithm.

    @inproceedings{lirolem27056,
           booktitle = {The Genetic and Evolutionary Computation Conference (GECCO 2017)},
               month = {July},
               title = {Deriving and improving CMA-ES with Information geometric trust regions},
              author = {Abbas Abdolmaleki and Bob Price and Nuno Lau and Luis Paulo Reis and Gerhard Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592e2810)},
                 url = {http://eprints.lincoln.ac.uk/27056/},
            abstract = {CMA-ES is one of the most popular stochastic search algorithms.
    It performs favourably in many tasks without the need of extensive
    parameter tuning. The algorithm has many beneficial properties,
    including automatic step-size adaptation, efficient covariance updates
    that incorporates the current samples as well as the evolution
    path and its invariance properties. Its update rules are composed
    of well established heuristics where the theoretical foundations of
    some of these rules are also well understood. In this paper we
    will fully derive all CMA-ES update rules within the framework of
    expectation-maximisation-based stochastic search algorithms using
    information-geometric trust regions. We show that the use of the trust
    region results in similar updates to CMA-ES for the mean and the
    covariance matrix while it allows for the derivation of an improved
    update rule for the step-size. Our new algorithm, Trust-Region Covariance
    Matrix Adaptation Evolution Strategy (TR-CMA-ES) is
    fully derived from first order optimization principles and performs
    favourably in compare to standard CMA-ES algorithm.}
    }
  • H. Abdulsamad, O. Arenz, J. Peters, and G. Neumann, “State-regularized policy search for linearized dynamical systems,” in Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS), 2017.
    [BibTeX] [Abstract] [EPrints]

    Trajectory-Centric Reinforcement Learning and Trajectory Optimization methods optimize a sequence of feedbackcontrollers by taking advantage of local approximations of model dynamics and cost functions. Stability of the policy update is a major issue for these methods, rendering them hard to apply for highly nonlinear systems. Recent approaches combine classical Stochastic Optimal Control methods with information-theoretic bounds to control the step-size of the policy update and could even be used to train nonlinear deep control policies. These methods bound the relative entropy between the new and the old policy to ensure a stable policy update. However, despite the bound in policy space, the state distributions of two consecutive policies can still differ significantly, rendering the used local approximate models invalid. To alleviate this issue we propose enforcing a relative entropy constraint not only on the policy update, but also on the update of the state distribution, around which the dynamics and cost are being approximated. We present a derivation of the closed-form policy update and show that our approach outperforms related methods on two nonlinear and highly dynamic simulated systems.

    @inproceedings{lirolem27055,
           booktitle = {Proceedings of the International Conference on Automated Planning and Scheduling (ICAPS)},
               month = {June},
               title = {State-regularized policy search for linearized dynamical systems},
              author = {Hany Abdulsamad and Oleg Arenz and Jan Peters and Gerhard Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592e23c0)},
                 url = {http://eprints.lincoln.ac.uk/27055/},
            abstract = {Trajectory-Centric Reinforcement Learning and Trajectory
    Optimization methods optimize a sequence of feedbackcontrollers
    by taking advantage of local approximations of
    model dynamics and cost functions. Stability of the policy update
    is a major issue for these methods, rendering them hard
    to apply for highly nonlinear systems. Recent approaches
    combine classical Stochastic Optimal Control methods with
    information-theoretic bounds to control the step-size of the
    policy update and could even be used to train nonlinear deep
    control policies. These methods bound the relative entropy
    between the new and the old policy to ensure a stable policy
    update. However, despite the bound in policy space, the
    state distributions of two consecutive policies can still differ
    significantly, rendering the used local approximate models invalid.
    To alleviate this issue we propose enforcing a relative
    entropy constraint not only on the policy update, but also on
    the update of the state distribution, around which the dynamics
    and cost are being approximated. We present a derivation
    of the closed-form policy update and show that our approach
    outperforms related methods on two nonlinear and highly dynamic
    simulated systems.}
    }
  • R. Akrour, D. Sorokin, J. Peters, and G. Neumann, “Local Bayesian optimization of motor skills,” in International Conference on Machine Learning (ICML), 2017.
    [BibTeX] [Abstract] [EPrints]

    Bayesian optimization is renowned for its sample efficiency but its application to higher dimensional tasks is impeded by its focus on global optimization. To scale to higher dimensional problems, we leverage the sample efficiency of Bayesian optimization in a local context. The optimization of the acquisition function is restricted to the vicinity of a Gaussian search distribution which is moved towards high value areas of the objective. The proposed informationtheoretic update of the search distribution results in a Bayesian interpretation of local stochastic search: the search distribution encodes prior knowledge on the optimum?s location and is weighted at each iteration by the likelihood of this location?s optimality. We demonstrate the effectiveness of our algorithm on several benchmark objective functions as well as a continuous robotic task in which an informative prior is obtained by imitation learning.

    @inproceedings{lirolem27902,
           booktitle = {International Conference on Machine Learning (ICML)},
               month = {August},
               title = {Local Bayesian optimization of motor skills},
              author = {R. Akrour and D. Sorokin and J. Peters and G. Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592c63e0)},
                 url = {http://eprints.lincoln.ac.uk/27902/},
            abstract = {Bayesian optimization is renowned for its sample
    efficiency but its application to higher dimensional
    tasks is impeded by its focus on global
    optimization. To scale to higher dimensional
    problems, we leverage the sample efficiency of
    Bayesian optimization in a local context. The
    optimization of the acquisition function is restricted
    to the vicinity of a Gaussian search distribution
    which is moved towards high value areas
    of the objective. The proposed informationtheoretic
    update of the search distribution results
    in a Bayesian interpretation of local stochastic
    search: the search distribution encodes prior
    knowledge on the optimum?s location and is
    weighted at each iteration by the likelihood of
    this location?s optimality. We demonstrate the
    effectiveness of our algorithm on several benchmark
    objective functions as well as a continuous
    robotic task in which an informative prior is obtained
    by imitation learning.}
    }
  • P. Baxter, E. Ashurst, R. Read, J. Kennedy, and T. Belpaeme, “Robot education peers in a situated primary school study: personalisation promotes child learning,” PLoS One, 2017.
    [BibTeX] [Abstract] [EPrints]

    The benefit of social robots to support child learning in an educational context over an extended period of time is evaluated. Specifically, the effect of personalisation and adaptation of robot social behaviour is assessed. Two autonomous robots were embedded within two matched classrooms of a primary school for a continuous two week period without experimenter supervision to act as learning companions for the children for familiar and novel subjects. Results suggest that while children in both personalised and non-personalised conditions learned, there was increased child learning of a novel subject exhibited when interacting with a robot that personalised its behaviours, with indications that this benefit extended to other class-based performance. Additional evidence was obtained suggesting that there is increased acceptance of the personalised robot peer over a non-personalised version. These results provide the first evidence in support of peer-robot behavioural personalisation having a positive influence on learning when embedded in a learning environment for an extended period of time.

    @article{lirolem27582,
               month = {May},
               title = {Robot education peers in a situated primary school study: personalisation promotes child learning},
              author = {Paul Baxter and Emily Ashurst and Robin Read and James Kennedy and Tony Belpaeme},
           publisher = {Public Library of Science},
                year = {2017},
             journal = {PLoS One},
            keywords = {ARRAY(0x7f78592c6098)},
                 url = {http://eprints.lincoln.ac.uk/27582/},
            abstract = {The benefit of social robots to support child learning in an educational context over an extended period of time is evaluated. Specifically, the effect of personalisation and adaptation of robot social behaviour is assessed. Two autonomous robots were embedded within two matched classrooms of a primary school for a continuous two week period without experimenter supervision to act as learning companions for the children for familiar and novel subjects. Results suggest that while children in both personalised and non-personalised conditions learned, there was increased child learning of a novel subject exhibited when interacting with a robot that personalised its behaviours, with indications that this benefit extended to other class-based performance. Additional evidence was obtained suggesting that there is increased acceptance of the personalised robot peer over a non-personalised version. These results provide the first evidence in support of peer-robot behavioural personalisation having a positive influence on learning when embedded in a learning environment for an extended period of time.}
    }
  • N. Bellotto, M. Fernandez-Carmona, and S. Cosar, “ENRICHME integration of ambient intelligence and robotics for AAL,” in Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (AAAI 2017 Spring Symposium), 2017.
    [BibTeX] [Abstract] [EPrints]

    Technological advances and affordability of recent smart sensors, as well as the consolidation of common software platforms for the integration of the latter and robotic sensors, are enabling the creation of complex active and assisted living environments for improving the quality of life of the elderly and the less able people. One such example is the integrated system developed by the European project ENRICHME, the aim of which is to monitor and prolong the independent living of old people affected by mild cognitive impairments with a combination of smart-home, robotics and web technologies. This paper presents in particular the design and technological solutions adopted to integrate, process and store the information provided by a set of fixed smart sensors and mobile robot sensors in a domestic scenario, including presence and contact detectors, environmental sensors, and RFID-tagged objects, for long-term user monitoring and

    @inproceedings{lirolem25362,
           booktitle = {Wellbeing AI: From Machine Learning to Subjectivity Oriented Computing (AAAI 2017 Spring Symposium)},
               month = {March},
               title = {ENRICHME integration of ambient intelligence and robotics for AAL},
              author = {Nicola Bellotto and Manuel Fernandez-Carmona and Serhan Cosar},
           publisher = {AAAI},
                year = {2017},
            keywords = {ARRAY(0x7f78592e2468)},
                 url = {http://eprints.lincoln.ac.uk/25362/},
            abstract = {Technological advances and affordability of recent smart sensors, as well as the consolidation of common software platforms for the integration of the latter and robotic sensors, are enabling the creation of complex active and assisted living environments for improving the quality of life of the elderly and the less able people. One such example is the integrated system developed by the European project ENRICHME, the aim of which is to monitor and prolong the independent living of old people affected by mild cognitive impairments with a combination of smart-home, robotics and web technologies. This paper presents in particular the design and technological solutions adopted to integrate, process and store the information provided by a set of fixed smart sensors and mobile robot sensors in a domestic scenario, including presence and contact detectors, environmental sensors, and RFID-tagged objects, for long-term user monitoring and}
    }
  • C. Coppola, S. Cosar, D. Faria, and N. Bellotto, “Automatic detection of human interactions from RGB-D data for social activity classification,” in IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2017.
    [BibTeX] [Abstract] [EPrints]

    We present a system for the temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, it is important to be able to detect temporally the intervals of time in which humans are performing an individual activity or a social one. Recognition of the human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities could be useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors able to represent the phenomena; (2) develop a computational model able to discern the intervals in which a pair of people are interacting or performing individual activities; (3) provide a public dataset with RGB-D videos where social interactions and individual activities happen in a continuous stream. Results show that using the proposed approach allows to reach a good performance in the temporal segmentation of social activities.

    @inproceedings{lirolem27647,
           booktitle = {IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)},
               title = {Automatic detection of human interactions from RGB-D data for social activity classification},
              author = {Claudio Coppola and Serhan Cosar and Diego Faria and Nicola Bellotto},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592c61d0)},
                 url = {http://eprints.lincoln.ac.uk/27647/},
            abstract = {We present a system for the temporal detection of social interactions. Many of the works until now have succeeded in recognising activities from clipped videos in datasets, but for robotic applications, it is important to be able to move to more realistic data. For this reason, it is important to be able to detect temporally the intervals of time in which humans are performing an individual activity or a social one. Recognition of the human activities is a key feature for analysing the human behaviour. In particular, recognition of social activities could be useful to trigger human-robot interactions or to detect situations of potential danger. Based on that, this research has three goals: (1) define a new set of descriptors able to represent the phenomena; (2) develop a computational model able to discern the intervals in which a pair of people are interacting or performing individual activities; (3) provide a public dataset with RGB-D videos where social interactions and individual activities happen in a continuous stream. Results show that using the proposed approach allows to reach a good performance in the temporal segmentation of social activities.}
    }
  • S. Cosar, C. Coppola, and N. Bellotto, “Volume-based human re-identification with RGB-D cameras,” in VISAPP – International Conference on Computer Vision Theory and Applications, 2017.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an RGB-D based human re-identification approach using novel biometrics features from the body’s volume. Existing work based on RGB images or skeleton features have some limitations for real-world robotic applications, most notably in dealing with occlusions and orientation of the user. Here, we propose novel features that allow performing re-identification when the person is facing side/backward or the person is partially occluded. The proposed approach has been tested for various scenarios including different views, occlusion and the public BIWI RGBD-ID dataset.

    @inproceedings{lirolem25360,
           booktitle = {VISAPP - International Conference on Computer Vision Theory and Applications},
               month = {February},
               title = {Volume-based human re-identification with RGB-D cameras},
              author = {Serhan Cosar and Claudio Coppola and Nicola Bellotto},
                year = {2017},
            keywords = {ARRAY(0x7f78593edae0)},
                 url = {http://eprints.lincoln.ac.uk/25360/},
            abstract = {This paper presents an RGB-D based human re-identification approach using novel biometrics features from the body's volume. Existing work based on RGB images or skeleton features have some limitations for real-world robotic applications, most notably in dealing with occlusions and orientation of the user. Here, we propose novel features that allow performing re-identification when the person is facing side/backward or the person is partially occluded. The proposed approach has been tested for various scenarios including different views, occlusion and the public BIWI RGBD-ID dataset.}
    }
  • H. Cuayahuitl, “Deep reinforcement learning for conversational robots playing games,” in IEEE RAS International Conference on Humanoid Robots, 2017.
    [BibTeX] [Abstract] [EPrints]

    Deep reinforcement learning for interactive multimodal robots is attractive for endowing machines with trainable skill acquisition. But this form of learning still represents several challenges. The challenge that we focus in this paper is effective policy learning. To address that, in this paper we compare the Deep Q-Networks (DQN) method against a variant that aims for stronger decisions than the original method by avoiding decisions with the lowest negative rewards. We evaluated our baseline and proposed algorithms in agents playing the game of Noughts and Crosses with two grid sizes (3×3 and 5×5). Experimental results show evidence that our proposed method can lead to more effective policies than the baseline DQN method, which can be used for training interactive social robots.

    @inproceedings{lirolem29060,
           booktitle = {IEEE RAS International Conference on Humanoid Robots},
               month = {November},
               title = {Deep reinforcement learning for conversational robots playing games},
              author = {Heriberto Cuayahuitl},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592c2f28)},
                 url = {http://eprints.lincoln.ac.uk/29060/},
            abstract = {Deep reinforcement learning for interactive multimodal robots is attractive for endowing machines with trainable skill acquisition. But this form of learning still represents several challenges. The challenge that we focus in this paper is effective policy learning. To address that, in this paper we compare the Deep Q-Networks (DQN) method against a variant that aims for stronger decisions than the original method by avoiding decisions with the lowest negative rewards. We evaluated our baseline and proposed algorithms in agents playing the game of Noughts and Crosses with two grid sizes (3x3 and 5x5). Experimental results show evidence that our proposed method can lead to more effective policies than the baseline DQN method, which can be used for training interactive social robots.}
    }
  • H. Cuayahuitl and S. Yu, “Deep reinforcement learning of dialogue policies with less weight updates,” in International Conference of the Speech Communication Association (INTERSPEECH), 2017.
    [BibTeX] [Abstract] [EPrints]

    Deep reinforcement learning dialogue systems are attractive because they can jointly learn their feature representations and policies without manual feature engineering. But its application is challenging due to slow learning. We propose a two-stage method for accelerating the induction of single or multi-domain dialogue policies. While the first stage reduces the amount of weight updates over time, the second stage uses very limited minibatches (of as much as two learning experiences) sampled from experience replay memories. The former frequently updates the weights of the neural nets at early stages of training, and decreases the amount of updates as training progresses by performing updates during exploration and by skipping updates during exploitation. The learning process is thus accelerated through less weight updates in both stages. An empirical evaluation in three domains (restaurants, hotels and tv guide) confirms that the proposed method trains policies 5 times faster than a baseline without the proposed method. Our findings are useful for training larger-scale neural-based spoken dialogue systems.

    @inproceedings{lirolem27676,
           booktitle = {International Conference of the Speech Communication Association (INTERSPEECH)},
               month = {August},
               title = {Deep reinforcement learning of dialogue policies with less weight updates},
              author = {Heriberto Cuayahuitl and Seunghak Yu},
                year = {2017},
            keywords = {ARRAY(0x7f78592ee6e8)},
                 url = {http://eprints.lincoln.ac.uk/27676/},
            abstract = {Deep reinforcement learning dialogue systems are attractive because they can jointly learn their feature representations and policies without manual feature engineering. But its application is challenging due to slow learning. We propose a two-stage method for accelerating the induction of single or multi-domain dialogue policies. While the first stage reduces the amount of weight updates over time, the second stage uses very limited minibatches (of as much as two learning experiences) sampled from experience replay memories. The former frequently updates the weights of the neural nets at early stages of training, and decreases the amount of updates as training progresses by performing updates during exploration and by skipping updates during exploitation. The learning process is thus accelerated
    through less weight updates in both stages. An empirical evaluation in three domains (restaurants, hotels and tv guide) confirms that the proposed method trains policies 5 times faster than a baseline without the proposed method. Our findings are useful for training larger-scale neural-based spoken dialogue systems.}
    }
  • H. Cuayahuitl, S. Yu, A. Williamson, and J. Carse, “Scaling up deep reinforcement learning for multi-domain dialogue systems,” in International Joint Conference on Neural Networks (IJCNN), 2017.
    [BibTeX] [Abstract] [EPrints]

    Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems due to large search spaces. This paper proposes a three-stage method for multi-domain dialogue policy learning–termed NDQN, and applies it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. In this method, the first stage does multi-policy learning via a network of DQN agents; the second makes use of compact state representations by compressing raw inputs; and the third stage applies a pre-training phase for bootstraping the behaviour of agents in the network. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that the proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems. An additional evaluation reports that the NDQN agents outperformed a K-Nearest Neighbour baseline in task success and dialogue length, yielding more efficient and successful dialogues.

    @inproceedings{lirolem26622,
           booktitle = {International Joint Conference on Neural Networks (IJCNN)},
               month = {May},
               title = {Scaling up deep reinforcement learning for multi-domain dialogue systems},
              author = {Heriberto Cuayahuitl and Seunghak Yu and Ashley Williamson and Jacob Carse},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592e2d38)},
                 url = {http://eprints.lincoln.ac.uk/26622/},
            abstract = {Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems due to large search spaces. This paper proposes a three-stage method for multi-domain dialogue policy learning{--}termed NDQN, and applies it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. In this method, the first stage does multi-policy learning via a network of DQN agents; the second makes use of compact state representations by compressing raw inputs; and the third stage applies a pre-training phase for bootstraping the behaviour of agents in the network. Experimental results comparing DQN
    (baseline) versus NDQN (proposed) using simulations report that the proposed method exhibits better scalability and is
    promising for optimising the behaviour of multi-domain dialogue systems. An additional evaluation reports that the NDQN agents outperformed a K-Nearest Neighbour baseline in task success and dialogue length, yielding more efficient and successful dialogues.}
    }
  • N. Dethlefs, M. Milders, H. Cuayáhuitl, T. Al-Salkini, and L. Douglas, “A natural language-based presentation of cognitive stimulation to people with dementia in assistive technology: a pilot study,” Informatics for Health and Social Care, pp. 1-12, 2017.
    [BibTeX] [Abstract] [EPrints]

    Currently, an estimated 36 million people worldwide are affected by Alzheimer?s disease or related dementias. In the absence of a cure, non-pharmacological interventions, such as cognitive stimulation, which slow down the rate of deterioration can benefit people with dementia and their caregivers. Such interventions have shown to improve well-being and slow down the rate of cognitive decline. It has further been shown that cognitive stimulation in interaction with a computer is as effective as with a human. However, the need to operate a computer often represents a difficulty for the elderly and stands in the way of widespread adoption. A possible solution to this obstacle is to provide a spoken natural language interface that allows people with dementia to interact with the cognitive stimulation software in the same way as they would interact with a human caregiver. This makes the assistive technology accessible to users regardless of their technical skills and provides a fully intuitive user experience. This article describes a pilot study that evaluated the feasibility of computer-based cognitive stimulation through a spoken natural language interface. Prototype software was evaluated with 23 users, including healthy elderly people and people with dementia. Feedback was overwhelmingly positive.

    @article{lirolem28284,
               month = {December},
               title = {A natural language-based presentation of cognitive stimulation to people with dementia in assistive technology: a pilot study},
              author = {Nina Dethlefs and Maarten Milders and Heriberto Cuay{\'a}huitl and Turkey Al-Salkini and Lorraine Douglas},
           publisher = {Taylor \& Francis: STM},
                year = {2017},
               pages = {1--12},
             journal = {Informatics for Health and Social Care},
            keywords = {ARRAY(0x7f78592d5330)},
                 url = {http://eprints.lincoln.ac.uk/28284/},
            abstract = {Currently, an estimated 36 million people worldwide are affected by Alzheimer?s disease or related dementias. In the absence of a cure, non-pharmacological interventions, such as cognitive stimulation, which slow down the rate of deterioration can benefit people with dementia and their caregivers. Such interventions have shown to improve well-being and slow down the rate of cognitive decline. It has further been shown that cognitive stimulation in interaction with a computer is as effective as with a human. However, the need to operate a computer often represents a difficulty for the elderly and stands in the way of widespread adoption. A possible solution to this obstacle is to provide a spoken natural language interface that allows people with dementia to interact with the cognitive stimulation software in the same way as they would interact with a human caregiver. This makes the assistive technology accessible to users regardless of their technical skills and provides a fully intuitive user experience. This article describes a pilot study that evaluated the feasibility of computer-based cognitive stimulation through a spoken natural language interface. Prototype software was evaluated with 23 users, including healthy elderly people and people with dementia. Feedback was overwhelmingly positive.}
    }
  • T. Duckett, A. Tapus, and N. Bellotto, “Editorial to special issue on the Seventh European Conference on Mobile Robots (ECMR?15),” Robotics and Autonomous Systems, vol. 91, p. 348, 2017.
    [BibTeX] [Abstract] [EPrints]

    This Special Issue is based on a selection of the best papers presented at the Seventh European Conference on Mobile Robots (ECMR?15), September 2nd?4th, 2015, in Lincoln, UK.

    @article{lirolem28034,
              volume = {91},
               month = {May},
              author = {Tom Duckett and Adriana Tapus and Nicola Bellotto},
               title = {Editorial to special issue on the Seventh European Conference on Mobile Robots (ECMR?15)},
           publisher = {Elsevier},
             journal = {Robotics and Autonomous Systems},
               pages = {348},
                year = {2017},
            keywords = {ARRAY(0x7f78592dd380)},
                 url = {http://eprints.lincoln.ac.uk/28034/},
            abstract = {This Special Issue is based on a selection of the best papers presented at the Seventh European Conference on Mobile Robots (ECMR?15), September 2nd?4th, 2015, in Lincoln, UK.}
    }
  • F. End, R. Akrour, J. Peters, and G. Neumann, “Layered direct policy search for learning hierarchical skills,” in International Conference on Robotics and Automation (ICRA), 2017.
    [BibTeX] [Abstract] [EPrints]

    Solutions to real world robotic tasks often require complex behaviors in high dimensional continuous state and action spaces. Reinforcement Learning (RL) is aimed at learning such behaviors but often fails for lack of scalability. To address this issue, Hierarchical RL (HRL) algorithms leverage hierarchical policies to exploit the structure of a task. However, many HRL algorithms rely on task specific knowledge such as a set of predefined sub-policies or sub-goals. In this paper we propose a new HRL algorithm based on information theoretic principles to autonomously uncover a diverse set of sub-policies and their activation policies. Moreover, the learning process mirrors the policys structure and is thus also hierarchical, consisting of a set of independent optimization problems. The hierarchical structure of the learning process allows us to control the learning rate of the sub-policies and the gating individually and add specific information theoretic constraints to each layer to ensure the diversification of the subpolicies. We evaluate our algorithm on two high dimensional continuous tasks and experimentally demonstrate its ability to autonomously discover a rich set of sub-policies.

    @inproceedings{lirolem26737,
           booktitle = {International Conference on Robotics and Automation (ICRA)},
               month = {May},
               title = {Layered direct policy search for learning hierarchical skills},
              author = {F. End and R. Akrour and J. Peters and G. Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592c2af0)},
                 url = {http://eprints.lincoln.ac.uk/26737/},
            abstract = {Solutions to real world robotic tasks often require
    complex behaviors in high dimensional continuous state and
    action spaces. Reinforcement Learning (RL) is aimed at learning
    such behaviors but often fails for lack of scalability. To
    address this issue, Hierarchical RL (HRL) algorithms leverage
    hierarchical policies to exploit the structure of a task. However,
    many HRL algorithms rely on task specific knowledge such
    as a set of predefined sub-policies or sub-goals. In this paper
    we propose a new HRL algorithm based on information
    theoretic principles to autonomously uncover a diverse set
    of sub-policies and their activation policies. Moreover, the
    learning process mirrors the policys structure and is thus also
    hierarchical, consisting of a set of independent optimization
    problems. The hierarchical structure of the learning process
    allows us to control the learning rate of the sub-policies and
    the gating individually and add specific information theoretic
    constraints to each layer to ensure the diversification of the subpolicies.
    We evaluate our algorithm on two high dimensional
    continuous tasks and experimentally demonstrate its ability to
    autonomously discover a rich set of sub-policies.}
    }
  • P. G. Esteban, P. Baxter, T. Belpaeme, E. Billing, H. Cai, H. Cao, M. Coeckelbergh, C. Costescu, D. David, A. D. Beir, Y. Fang, Z. Ju, J. Kennedy, H. Liu, A. Mazel, A. Pandey, K. Richardson, E. Senft, S. Thill, G. V. de Perre, B. Vanderborght, D. Vernon, H. Yu, and T. Ziemke, “How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder,” Paladyn, Journal of Behavioral Robotics, vol. 8, iss. 1, 2017.
    [BibTeX] [Abstract] [EPrints]

    Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms.However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy.We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.

    @article{lirolem27519,
              volume = {8},
              number = {1},
               month = {May},
              author = {Pablo G. Esteban and Paul Baxter and Tony Belpaeme and Erik Billing and Haibin Cai and Hoang-Long Cao and Mark Coeckelbergh and Cristina Costescu and Daniel David and Albert De Beir and Yinfeng Fang and Zhaojie Ju and James Kennedy and Honghai Liu and Alexandre Mazel and Amit Pandey and Kathleen Richardson and Emmanue Senft and Serge Thill and Greet Van de Perre and Bram Vanderborght and David Vernon and Hui Yu and Tom Ziemke},
               title = {How to build a supervised autonomous system for robot-enhanced therapy for children with autism spectrum disorder},
           publisher = {Springer/Versita with DeGruyter},
             journal = {Paladyn, Journal of Behavioral Robotics},
                year = {2017},
            keywords = {ARRAY(0x7f78592e0630)},
                 url = {http://eprints.lincoln.ac.uk/27519/},
            abstract = {Robot-Assisted Therapy (RAT) has successfully been used to improve social skills in children with autism spectrum disorders (ASD) through remote control of the robot in so-called Wizard of Oz (WoZ) paradigms.However, there is a need to increase the autonomy of the robot both to lighten the burden on human therapists (who have to remain in control and, importantly, supervise the robot) and to provide a consistent therapeutic experience. This paper seeks to provide insight into increasing the autonomy level of social robots in therapy to move beyond WoZ. With the final aim of improved human-human social interaction for the children, this multidisciplinary research seeks to facilitate the use of social robots as tools in clinical situations by addressing the challenge of increasing robot autonomy.We introduce the clinical framework in which the developments are tested, alongside initial data obtained from patients in a first phase of the project using a WoZ set-up mimicking the targeted supervised-autonomy behaviour. We further describe the implemented system architecture capable of providing the robot with supervised autonomy.}
    }
  • F. B. Farraj, T. Osa, N. Pedemonte, J. Peters, G. Neumann, and P. R. Giordano, “A learning-based shared control architecture for interactive task execution,” in IEEE International Conference on Robotics and Automation (ICRA), 2017.
    [BibTeX] [Abstract] [EPrints]

    Shared control is a key technology for various robotic applications in which a robotic system and a human operator are meant to collaborate efficiently. In order to achieve efficient task execution in shared control, it is essential to predict the desired behavior for a given situation or context to simplify the control task for the human operator. To do this prediction, we use Learning from Demonstration (LfD), which is a popular approach for transferring human skills to robots. We encode the demonstrated behavior as trajectory distributions and generalize the learned distributions to new situations. The goal of this paper is to present a shared control framework that uses learned expert distributions to gain more autonomy. Our approach controls the balance between the controller?s autonomy and the human preference based on the distributions of the demonstrated trajectories. Moreover, the learned distributions are autonomously refined from collaborative task executions, resulting in a master-slave system with increasing autonomy that requires less user input with an increasing number of task executions. We experimentally validated that our shared control approach enables efficient task executions. Moreover, the conducted experiments demonstrated that the developed system improves its performances through interactive task executions with our shared control.

    @inproceedings{lirolem26738,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
               month = {May},
               title = {A learning-based shared control architecture for interactive task execution},
              author = {F. B. Farraj and T. Osa and N. Pedemonte and J. Peters and G. Neumann and P. R. Giordano},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592d5108)},
                 url = {http://eprints.lincoln.ac.uk/26738/},
            abstract = {Shared control is a key technology for various
    robotic applications in which a robotic system and a human
    operator are meant to collaborate efficiently. In order to achieve
    efficient task execution in shared control, it is essential to
    predict the desired behavior for a given situation or context
    to simplify the control task for the human operator. To do this
    prediction, we use Learning from Demonstration (LfD), which is
    a popular approach for transferring human skills to robots. We
    encode the demonstrated behavior as trajectory distributions
    and generalize the learned distributions to new situations. The
    goal of this paper is to present a shared control framework
    that uses learned expert distributions to gain more autonomy.
    Our approach controls the balance between the controller?s
    autonomy and the human preference based on the distributions
    of the demonstrated trajectories. Moreover, the learned
    distributions are autonomously refined from collaborative task
    executions, resulting in a master-slave system with increasing
    autonomy that requires less user input with an increasing
    number of task executions. We experimentally validated that
    our shared control approach enables efficient task executions.
    Moreover, the conducted experiments demonstrated that the
    developed system improves its performances through interactive
    task executions with our shared control.}
    }
  • M. Fernandez-Carmona, S. Cosar, C. Coppola, and N. Bellotto, “Entropy-based abnormal activity detection fusing RGB-D and domotic sensors,” in IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), 2017.
    [BibTeX] [Abstract] [EPrints]

    The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) with those ones equipping modern mobile robots (e.g. RGBD camera) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGBD camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment?s area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with RGBD data and a comprehensive domotic dataset containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and the potential for complex anomaly detection in AAL settings.

    @inproceedings{lirolem28779,
           booktitle = {IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI)},
               month = {November},
               title = {Entropy-based abnormal activity detection fusing RGB-D and domotic sensors},
              author = {Manuel Fernandez-Carmona and Serhan Cosar and Claudio Coppola and Nicola Bellotto},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592e4c60)},
                 url = {http://eprints.lincoln.ac.uk/28779/},
            abstract = {The automatic detection of anomalies in Active and Assisted Living (AAL) environments is important for monitoring the wellbeing and safety of the elderly at home. The integration of smart domotic sensors (e.g. presence detectors) with those ones equipping modern mobile robots (e.g. RGBD camera) provides new opportunities for addressing this challenge. In this paper, we propose a novel solution to combine local activity levels detected by a single RGBD camera with the global activity perceived by a network of domotic sensors. Our approach relies on a new method for computing such a global activity using various presence detectors, based on the concept of entropy from information theory. This entropy effectively shows how active a particular room or environment?s area is. The solution includes also a new application of Hybrid Markov Logic Networks (HMLNs) to merge different information sources for local and global anomaly detection. The system has been tested with RGBD data and a comprehensive domotic dataset containing data entries from 37 different domotic sensors (presence, temperature, light, energy consumption, door contact), which is made publicly available. The experimental results show the effectiveness of our approach and the potential for complex anomaly detection in AAL settings.}
    }
  • Q. Fu and S. Yue, “Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network,” in IEEE Int. Conf. on Robotics and Biomimetics, 2017.
    [BibTeX] [Abstract] [EPrints]

    How do animals, e.g. insects, detect meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? This open question has been very attractive for decades. In this paper, with respect to latest biological research progress made on motion detection circuitry, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely motion and position pathways explored in fly visual system, for mimicking the tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information on moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) Compared to conventional computer vision techniques, it turns out the computational simplicity of this model may benefit the utility in small robots for real time fixating. (2) The hybrid neural network structure fulfills the characteristics of a putative signal tuning map in physiology. (3) It also satisfies with a profound implication proposed by biologists: visual fixation behaviors could be simply tuned via only the position pathway; nevertheless, the motion-detecting pathway enhances the tracking precision.

    @inproceedings{lirolem28879,
           booktitle = {IEEE Int. Conf. on Robotics and Biomimetics},
               month = {December},
               title = {Mimicking fly motion tracking and fixation behaviors with a hybrid visual neural network},
              author = {Qinbing Fu and Shigang Yue},
                year = {2017},
            keywords = {ARRAY(0x7f78592e4ca8)},
                 url = {http://eprints.lincoln.ac.uk/28879/},
            abstract = {How do animals, e.g. insects, detect meaningful visual motion cues involving directional and locational information of moving objects in visual clutter accurately and efficiently? This open question has been very attractive for decades. In this paper, with respect to latest biological research progress made on motion detection circuitry, we conduct a novel hybrid visual neural network, combining the functionality of two bio-plausible, namely motion and position pathways explored in fly visual system, for mimicking the tracking and fixation behaviors. This modeling study extends a former direction selective neurons model to the higher level of behavior. The motivated algorithms can be used to guide a system that extracts location information on moving objects in a scene regardless of background clutter, using entirely low-level visual processing. We tested it against translational movements in synthetic and real-world scenes. The results demonstrated the following contributions: (1) Compared to conventional computer vision techniques, it turns out the computational simplicity of this model may benefit the utility in small robots for real time fixating. (2) The hybrid neural network structure fulfills the characteristics of a putative signal tuning map in physiology. (3) It also satisfies with a profound implication proposed by biologists: visual fixation behaviors could be simply tuned via only the position pathway; nevertheless, the motion-detecting pathway enhances the tracking precision.}
    }
  • Q. Fu and S. Yue, “Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background,” in The 2017 International Joint Conference on Neural Networks (IJCNN 2017), 2017.
    [BibTeX] [Abstract] [EPrints]

    The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is ?ltered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: ?rst, the neural network ful?lled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.

    @inproceedings{lirolem26619,
           booktitle = {The 2017 International Joint Conference on Neural Networks (IJCNN 2017)},
               month = {May},
               title = {Modeling direction selective visual neural network with ON and OFF pathways for extracting motion cues from cluttered background},
              author = {Qinbing Fu and Shigang Yue},
                year = {2017},
            keywords = {ARRAY(0x7f78592da438)},
                 url = {http://eprints.lincoln.ac.uk/26619/},
            abstract = {The nature endows animals robustvision systems for extracting and recognizing differentmotion cues, detectingpredators, chasing preys/mates in dynamic and cluttered environments. Direction selective neurons (DSNs), with preference to certain orientation visual stimulus, have been found in both vertebrates and invertebrates for decades. In thispaper, with respectto recent biological research progress in motion-detecting circuitry, we propose a novel way to model DSNs for recognizing movements on four cardinal directions. It is based on an architecture of ON and OFF visual pathways underlies a theory of splitting motion signals into parallel channels, encoding brightness increments and decrements separately. To enhance the edge selectivity and speed response to moving objects, we put forth a bio-plausible spatial-temporal network structure with multiple connections of same polarity ON/OFF cells. Each pair-wised combination is ?ltered with dynamic delay depending on sampling distance. The proposed vision system was challenged against image streams from both synthetic and cluttered real physical scenarios. The results demonstrated three major contributions: ?rst, the neural network ful?lled the characteristics of a postulated physiological map of conveying visual information through different neuropile layers; second, the DSNs model can extract useful directional motion cues from cluttered background robustly and timely, which hits at potential of quick implementation in visionbased micro mobile robots; moreover, it also represents better speed response compared to a state-of-the-art elementary motion detector.}
    }
  • Q. Fu, C. Hu, T. Liu, and S. Yue, “Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2017.
    [BibTeX] [Abstract] [EPrints]

    The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts’ visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been built for perceiving potential collisions in an efficient and reliable manner; a few modeling works have also demonstrated their effectiveness for robotic implementations. In this research, for the first time, we set up binocular neuronal models, combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions: (1) The arena tests involving multiple robots verified the robustness and efficiency of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models fulfilling corresponded biological research results. (3) The low-cost robot may also shed lights on similar bio-inspired embedded vision systems and swarm robotics applications.

    @inproceedings{lirolem27834,
           booktitle = {2017 IEEE/RSJ International Conference on Intelligent Robots and Systems},
               month = {September},
               title = {Collision selective LGMDs neuron models research benefits from a vision-based autonomous micro robot},
              author = {Qinbing Fu and Cheng Hu and Tian Liu and Shigang Yue},
                year = {2017},
            keywords = {ARRAY(0x7f78592ee6b8)},
                 url = {http://eprints.lincoln.ac.uk/27834/},
            abstract = {The developments of robotics inform research across a broad range of disciplines. In this paper, we will study and compare two collision selective neuron models via a vision-based autonomous micro robot. In the locusts' visual brain, two Lobula Giant Movement Detectors (LGMDs), i.e. LGMD1 and LGMD2, have been identified as looming sensitive neurons responding to rapidly expanding objects, yet with different collision selectivity. Both neurons have been built for perceiving potential collisions in an efficient and reliable manner; a few modeling works have also demonstrated their effectiveness for robotic implementations. In this research, for the first time, we set up binocular neuronal models, combining the functionalities of LGMD1 and LGMD2 neurons, in the visual modality of a ground mobile robot. The results of systematic on-line experiments demonstrated three contributions: (1) The arena tests involving multiple robots verified the robustness and efficiency of a reactive motion control strategy via integrating a bilateral pair of LGMD1 and LGMD2 models for collision detection in dynamic scenarios. (2) We pinpointed the different collision selectivity between LGMD1 and LGMD2 neuron models fulfilling corresponded biological research results. (3) The low-cost robot may also shed lights on similar bio-inspired embedded vision systems and swarm robotics applications.}
    }
  • A. Gabriel, R. Akrour, J. Peters, and G. Neumann, “Empowered skills,” in International Conference on Robotics and Automation (ICRA), 2017.
    [BibTeX] [Abstract] [EPrints]

    Robot Reinforcement Learning (RL) algorithms return a policy that maximizes a global cumulative reward signal but typically do not create diverse behaviors. Hence, the policy will typically only capture a single solution of a task. However, many motor tasks have a large variety of solutions and the knowledge about these solutions can have several advantages. For example, in an adversarial setting such as robot table tennis, the lack of diversity renders the behavior predictable and hence easy to counter for the opponent. In an interactive setting such as learning from human feedback, an emphasis on diversity gives the human more opportunity for guiding the robot and to avoid the latter to be stuck in local optima of the task. In order to increase diversity of the learned behaviors, we leverage prior work on intrinsic motivation and empowerment. We derive a new intrinsic motivation signal by enriching the description of a task with an outcome space, representing interesting aspects of a sensorimotor stream. For example, in table tennis, the outcome space could be given by the return position and return ball speed. The intrinsic motivation is now given by the diversity of future outcomes, a concept also known as empowerment. We derive a new policy search algorithm that maximizes a trade-off between the extrinsic reward and this intrinsic motivation criterion. Experiments on a planar reaching task and simulated robot table tennis demonstrate that our algorithm can learn a diverse set of behaviors within the area of interest of the tasks.

    @inproceedings{lirolem26736,
           booktitle = {International Conference on Robotics and Automation (ICRA)},
               month = {May},
               title = {Empowered skills},
              author = {A. Gabriel and R. Akrour and J. Peters and G. Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592e6c60)},
                 url = {http://eprints.lincoln.ac.uk/26736/},
            abstract = {Robot Reinforcement Learning (RL) algorithms
    return a policy that maximizes a global cumulative reward
    signal but typically do not create diverse behaviors. Hence, the
    policy will typically only capture a single solution of a task.
    However, many motor tasks have a large variety of solutions
    and the knowledge about these solutions can have several
    advantages. For example, in an adversarial setting such as
    robot table tennis, the lack of diversity renders the behavior
    predictable and hence easy to counter for the opponent. In an
    interactive setting such as learning from human feedback, an
    emphasis on diversity gives the human more opportunity for
    guiding the robot and to avoid the latter to be stuck in local
    optima of the task. In order to increase diversity of the learned
    behaviors, we leverage prior work on intrinsic motivation and
    empowerment. We derive a new intrinsic motivation signal by
    enriching the description of a task with an outcome space,
    representing interesting aspects of a sensorimotor stream. For
    example, in table tennis, the outcome space could be given
    by the return position and return ball speed. The intrinsic
    motivation is now given by the diversity of future outcomes,
    a concept also known as empowerment. We derive a new
    policy search algorithm that maximizes a trade-off between
    the extrinsic reward and this intrinsic motivation criterion.
    Experiments on a planar reaching task and simulated robot
    table tennis demonstrate that our algorithm can learn a diverse
    set of behaviors within the area of interest of the tasks.}
    }
  • G. H. W. Gebhardt, K. Daun, M. Schnaubelt, A. Hendrich, D. Kauth, and G. Neumann, “Learning to assemble objects with a robot swarm,” in Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS 17), 2017, pp. 1547-1549.
    [BibTeX] [Abstract] [EPrints]

    Large populations of simple robots can solve complex tasks, but controlling them is still a challenging problem, due to limited communication and computation power. In order to assemble objects, have shown that a human controller can solve such a task. Instead, we investigate how to learn the assembly of multiple objects with a single central controller. We propose splitting the assembly process in two sub-tasks — generating a top-level assembly policy and learning an object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.The resulting system is able to solve assembly tasks with varying object shapes being assembled as shown in multiple simulation scenarios.

    @inproceedings{lirolem28089,
               month = {May},
              author = {Gregor H. W. Gebhardt and Kevin Daun and Marius Schnaubelt and Alexander Hendrich and Daniel Kauth and Gerhard Neumann},
                note = {Extended abstract},
           booktitle = {Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems (AAMAS 17)},
               title = {Learning to assemble objects with a robot swarm},
           publisher = {international foundation for autonomous agents and multiagent systems},
               pages = {1547--1549},
                year = {2017},
            keywords = {ARRAY(0x7f78592dfef8)},
                 url = {http://eprints.lincoln.ac.uk/28089/},
            abstract = {Large populations of simple robots can solve complex tasks, but controlling them is still a challenging problem, due to limited communication and computation power. In order to assemble objects, have shown that a human controller can solve such a task. Instead, we investigate how to learn the assembly of multiple objects with a single central controller. We propose splitting the assembly process in two sub-tasks -- generating a top-level assembly policy and learning an object movement policy. The assembly policy plans the trajectories for each object and the object movement policy controls the trajectory execution.The resulting system is able to solve assembly tasks with varying object shapes being assembled as shown in multiple simulation scenarios.}
    }
  • G. H. W. Gebhardt, A. Kupcsik, and G. Neumann, “The kernel Kalman rule: efficient nonparametric inference with recursive least squares,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017.
    [BibTeX] [Abstract] [EPrints]

    Nonparametric inference techniques provide promising tools for probabilistic reasoning in high-dimensional nonlinear systems. Most of these techniques embed distributions into reproducing kernel Hilbert spaces (RKHS) and rely on the kernel Bayes? rule (KBR) to manipulate the embeddings. However, the computational demands of the KBR scale poorly with the number of samples and the KBR often suffers from numerical instabilities. In this paper, we present the kernel Kalman rule (KKR) as an alternative to the KBR. The derivation of the KKR is based on recursive least squares, inspired by the derivation of the Kalman innovation update. We apply the KKR to filtering tasks where we use RKHS embeddings to represent the belief state, resulting in the kernel Kalman filter (KKF). We show on a nonlinear state estimation task with high dimensional observations that our approach provides a significantly improved estimation accuracy while the computational demands are significantly decreased.

    @inproceedings{lirolem26739,
           booktitle = {Thirty-First AAAI Conference on Artificial Intelligence},
               month = {February},
               title = {The kernel Kalman rule: efficient nonparametric inference with recursive least squares},
              author = {G. H. W. Gebhardt and A. Kupcsik and G. Neumann},
           publisher = {AAAI},
                year = {2017},
            keywords = {ARRAY(0x7f78593edb70)},
                 url = {http://eprints.lincoln.ac.uk/26739/},
            abstract = {Nonparametric inference techniques provide promising tools
    for probabilistic reasoning in high-dimensional nonlinear systems.
    Most of these techniques embed distributions into reproducing
    kernel Hilbert spaces (RKHS) and rely on the kernel
    Bayes? rule (KBR) to manipulate the embeddings. However,
    the computational demands of the KBR scale poorly
    with the number of samples and the KBR often suffers from
    numerical instabilities. In this paper, we present the kernel
    Kalman rule (KKR) as an alternative to the KBR. The derivation
    of the KKR is based on recursive least squares, inspired
    by the derivation of the Kalman innovation update. We apply
    the KKR to filtering tasks where we use RKHS embeddings
    to represent the belief state, resulting in the kernel Kalman filter
    (KKF). We show on a nonlinear state estimation task with
    high dimensional observations that our approach provides a
    significantly improved estimation accuracy while the computational
    demands are significantly decreased.}
    }
  • M. Hanheide, M. Göbelbecker, G. S. Horn, A. Pronobis, K. Sjöö, A. Aydemir, P. Jensfelt, C. Gretton, R. Dearden, M. Janicek, H. Zender, G. Kruijff, N. Hawes, and J. L. Wyatt, “Robot task planning and explanation in open and uncertain worlds,” Artificial Intelligence, 2017.
    [BibTeX] [Abstract] [EPrints]

    A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot’s knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot’s actions can have: epistemic effects (I believe X because I saw it) and assumptions (I’ll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.

    @article{lirolem18592,
               month = {December},
               title = {Robot task planning and explanation in open and uncertain worlds},
              author = {Marc Hanheide and Moritz G{\"o}belbecker and Graham S. Horn and Andrzej Pronobis and Kristoffer Sj{\"o}{\"o} and Alper Aydemir and Patric Jensfelt and Charles Gretton and Richard Dearden and Miroslav Janicek and Hendrik Zender and Geert-Jan Kruijff and Nick Hawes and Jeremy L. Wyatt},
           publisher = {Elsevier},
                year = {2017},
             journal = {Artificial Intelligence},
            keywords = {ARRAY(0x7f78592c2df0)},
                 url = {http://eprints.lincoln.ac.uk/18592/},
            abstract = {A long-standing goal of AI is to enable robots to plan in the face of uncertain and incomplete information, and to handle task failure intelligently. This paper shows how to achieve this. There are two central ideas. The first idea is to organize the robot's knowledge into three layers: instance knowledge at the bottom, commonsense knowledge above that, and diagnostic knowledge on top. Knowledge in a layer above can be used to modify knowledge in the layer(s) below. The second idea is that the robot should represent not just how its actions change the world, but also what it knows or believes. There are two types of knowledge effects the robot's actions can have: epistemic effects (I believe X because I saw it) and assumptions (I'll assume X to be true). By combining the knowledge layers with the models of knowledge effects, we can simultaneously solve several problems in robotics: (i) task planning and execution under uncertainty; (ii) task planning and execution in open worlds; (iii) explaining task failure; (iv) verifying those explanations. The paper describes how the ideas are implemented in a three-layer architecture on a mobile robot platform. The robot implementation was evaluated in five different experiments on object search, mapping, and room categorization.}
    }
  • M. Hanheide, D. Hebesberger, and T. Krajnik, “The when, where, and how: an adaptive robotic info-terminal for care home residents ? a long-term study,” in Int. Conf. on Human-Robot Interaction (HRI), Vienna, 2017.
    [BibTeX] [Abstract] [EPrints]

    Adapting to users’ intentions is a key requirement for autonomous robots in general, and in care settings in particular. In this paper, a comprehensive long-term study of a mobile robot providing information services to residents, visitors, and staff of a care home is presented with a focus on adapting to the when and where the robot should be offering its services to best accommodate the users’ needs. Rather than providing a fixed schedule, the presented system takes the opportunity of long-term deployment to explore the space of possibilities of interaction while concurrently exploiting the model learned to provide better services. But in order to provide effective services to users in a care home, not only then when and where are relevant, but also the way how the information is provided and accessed. Hence, also the usability of the deployed system is studied specifically, in order to provide a most comprehensive overall assessment of a robotic info-terminal implementation in a care setting. Our results back our hypotheses, (i) that learning a spatiotemporal model of users’ intentions improves efficiency and usefulness of the system, and (ii) that the specific information sought after is indeed dependent on the location the info-terminal is offered.

    @inproceedings{lirolem25866,
           booktitle = {Int. Conf. on Human-Robot Interaction (HRI)},
               month = {March},
               title = {The when, where, and how: an adaptive robotic info-terminal for care home residents ? a long-term study},
              author = {Marc Hanheide and Denise Hebesberger and Tomas Krajnik},
             address = {Vienna},
           publisher = {ACM/IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592c3360)},
                 url = {http://eprints.lincoln.ac.uk/25866/},
            abstract = {Adapting to users' intentions is a key requirement for autonomous robots in general, and in care settings in particular. In this paper, a comprehensive long-term study of a mobile robot providing information services to residents, visitors, and staff of a care home is presented with a focus on adapting to the when and where the robot should be offering its services to best accommodate the users' needs. Rather than providing a fixed schedule, the presented system takes the opportunity of long-term deployment to explore the space of possibilities of interaction while concurrently exploiting the model learned to provide better services. But in order to provide effective services to users in a care home, not only then when and where are relevant, but also the way how the information is provided and accessed. Hence, also the usability of the deployed system is studied specifically, in order to provide a most comprehensive overall assessment of a robotic info-terminal implementation in a care setting. Our results back our hypotheses, (i) that learning a spatiotemporal model of users' intentions improves efficiency and usefulness of the system, and (ii) that the specific information sought after is indeed dependent on the location the info-terminal is offered.}
    }
  • D. Hebesberger, C. Dondrup, C. Gisinger, and M. Hanheide, “Patterns of use: how older adults with progressed dementia interact with a robot,” in Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI) Late Breaking Reports, Vienna, 2017.
    [BibTeX] [Abstract] [EPrints]

    Older adults represent a new user group of robots that are deployed in their private homes or in care facilities. In the presented study tangible aspects of older adults’ interaction with an autonomous robot were focused. The robot was deployed as a companion in physical therapy for older adults with progressed dementia. Interaction was possible via a mounted touch screen. The menu was structured in a single layer and icons were big and with strong contrast. Employing a detailed observation protocol, interaction frequencies and contexts were assessed. Thereby, it was found that most of the interaction was encouraged by the therapists and that two out of 12 older adults with progressed dementia showed self-inducted interactions.

    @inproceedings{lirolem25867,
           booktitle = {Proc ACM/IEEE Int. Conf. on Human-Robot Interaction (HRI) Late Breaking Reports},
               month = {March},
               title = {Patterns of use: how older adults with progressed dementia interact with a robot},
              author = {Denise Hebesberger and Christian Dondrup and Christoph Gisinger and Marc Hanheide},
             address = {Vienna},
           publisher = {ACM/IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592c2bb0)},
                 url = {http://eprints.lincoln.ac.uk/25867/},
            abstract = {Older adults represent a new user group of robots that are deployed in their private homes or in care facilities. In the presented study tangible aspects of older adults' interaction with an autonomous robot were focused. The robot was deployed as a companion in physical therapy for older adults with progressed dementia. Interaction was possible via a mounted touch screen. The menu was structured in a single layer and icons were big and with strong contrast. Employing a detailed observation protocol, interaction frequencies and contexts were assessed. Thereby, it was found that most of the interaction was encouraged by the therapists and that two out of 12 older adults with progressed dementia showed self-inducted interactions.}
    }
  • H. van Hoof, G. Neumann, and J. Peters, “Non-parametric policy search with limited information loss,” Journal of Machine Learning Research, 2017.
    [BibTeX] [Abstract] [EPrints]

    Learning complex control policies from non-linear and redundant sensory input is an important challenge for reinforcement learning algorithms. Non-parametric methods that approximate values functions or transition models can address this problem, by adapting to the complexity of the dataset. Yet, many current non-parametric approaches rely on unstable greedy maximization of approximate value functions, which might lead to poor convergence or oscillations in the policy update. A more robust policy update can be obtained by limiting the information loss between successive state-action distributions. In this paper, we develop a policy search algorithm with policy updates that are both robust and non-parametric. Our method can learn non-parametric control policies for infinite horizon continuous Markov decision processes with non-linear and redundant sensory representations. We investigate how we can use approximations of the kernel function to reduce the time requirements of the demanding non-parametric computations. In our experiments, we show the strong performance of the proposed method, and how it can be approximated effi- ciently. Finally, we show that our algorithm can learn a real-robot underpowered swing-up task directly from image data.

    @article{lirolem28020,
               month = {December},
               title = {Non-parametric policy search with limited information loss},
              author = {Herke van Hoof and Gerhard Neumann and Jan Peters},
           publisher = {Journal of Machine Learning Research},
                year = {2017},
             journal = {Journal of Machine Learning Research},
            keywords = {ARRAY(0x7f78592e6438)},
                 url = {http://eprints.lincoln.ac.uk/28020/},
            abstract = {Learning complex control policies from non-linear and redundant sensory input is an important
    challenge for reinforcement learning algorithms. Non-parametric methods that
    approximate values functions or transition models can address this problem, by adapting
    to the complexity of the dataset. Yet, many current non-parametric approaches rely on
    unstable greedy maximization of approximate value functions, which might lead to poor
    convergence or oscillations in the policy update. A more robust policy update can be obtained
    by limiting the information loss between successive state-action distributions. In this
    paper, we develop a policy search algorithm with policy updates that are both robust and
    non-parametric. Our method can learn non-parametric control policies for infinite horizon
    continuous Markov decision processes with non-linear and redundant sensory representations.
    We investigate how we can use approximations of the kernel function to reduce the
    time requirements of the demanding non-parametric computations. In our experiments, we
    show the strong performance of the proposed method, and how it can be approximated effi-
    ciently. Finally, we show that our algorithm can learn a real-robot underpowered swing-up
    task directly from image data.}
    }
  • B. Hu, S. Yue, and Z. Zhang, “A rotational motion perception neural network based on asymmetric spatiotemporal visual information processing,” IEEE Transactions on Neural Networks and Learning Systems, 2017.
    [BibTeX] [Abstract] [EPrints]

    All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.

    @article{lirolem24936,
               month = {December},
               title = {A rotational motion perception neural network based on asymmetric spatiotemporal visual information processing},
              author = {Bin Hu and Shigang Yue and Zhuhong Zhang},
           publisher = {IEEE},
                year = {2017},
             journal = {IEEE Transactions on Neural Networks and Learning Systems},
            keywords = {ARRAY(0x7f78592bf388)},
                 url = {http://eprints.lincoln.ac.uk/24936/},
            abstract = {All complex motion patterns can be decomposed into several elements, including translation, expansion/contraction, and rotational motion. In biological vision systems, scientists have found that specific types of visual neurons have specific preferences to each of the three motion elements. There are computational models on translation and expansion/contraction perceptions; however, little has been done in the past to create computational models for rotational motion perception. To fill this gap, we proposed a neural network that utilizes a specific spatiotemporal arrangement of asymmetric lateral inhibited direction selective neural networks (DSNNs) for rotational motion perception. The proposed neural network consists of two parts-presynaptic and postsynaptic parts. In the presynaptic part, there are a number of lateral inhibited DSNNs to extract directional visual cues. In the postsynaptic part, similar to the arrangement of the directional columns in the cerebral cortex, these direction selective neurons are arranged in a cyclic order to perceive rotational motion cues. In the postsynaptic network, the delayed excitation from each direction selective neuron is multiplied by the gathered excitation from this neuron and its unilateral counterparts depending on which rotation, clockwise (cw) or counter-cw (ccw), to perceive. Systematic experiments under various conditions and settings have been carried out and validated the robustness and reliability of the proposed neural network in detecting cw or ccw rotational motion. This research is a critical step further toward dynamic visual information processing.}
    }
  • S. Keizer, M. Guhe, H. Cuayahuitl, I. Efstathiou, K. Engelbrecht, M. Dobre, A. Lascarides, and O. Lemon, “Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents,” in 15th Conference of the European chapter of the Association for Computational Linguistics, 2017.
    [BibTeX] [Abstract] [EPrints]

    In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game ?Settlers of Catan?. The comparison is based on human subjects playing games against artificial game-playing agents (?bots?) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators.

    @inproceedings{lirolem26621,
           booktitle = {15th Conference of the European chapter of the Association for Computational Linguistics},
               month = {April},
               title = {Evaluating persuasion strategies and deep reinforcement learning methods for negotiation dialogue agents},
              author = {Simon Keizer and Markus Guhe and Heriberto Cuayahuitl and Ioannis Efstathiou and Klaus-Peter Engelbrecht and Mihai Dobre and Alex Lascarides and Oliver Lemon},
           publisher = {ACL},
                year = {2017},
            keywords = {ARRAY(0x7f78592b73e0)},
                 url = {http://eprints.lincoln.ac.uk/26621/},
            abstract = {In this paper we present a comparative evaluation of various negotiation strategies within an online version of the game ?Settlers of Catan?. The comparison is based on human subjects playing games against artificial game-playing
    agents (?bots?) which implement different negotiation dialogue strategies, using a chat dialogue interface to negotiate trades. Our results suggest that a negotiation strategy that uses persuasion, as well as a strategy that is trained from data using Deep Reinforcement Learning, both lead to an improved win rate against humans, compared to previous rule-based and supervised learning baseline dialogue negotiators.}
    }
  • J. Kennedy, P. Baxter, and T. Belpaeme, “Nonverbal immediacy as a characterisation of social behaviour for human-robot interaction,” International Journal of Social Robotics, vol. 9, iss. 1, pp. 109-128, 2017.
    [BibTeX] [Abstract] [EPrints]

    An increasing amount of research has started to explore the impact of robot social behaviour on the outcome of a goal for a human interaction partner, such as cognitive learning gains. However, it remains unclear from what principles the social behaviour for such robots should be derived. Human models are often used, but in this paper an alternative approach is proposed. First, the concept of nonverbal immediacy from the communication literature is introduced, with a focus on how it can provide a characterisation of social behaviour, and the subsequent outcomes of such behaviour. A literature review is conducted to explore the impact on learning of the social cues which form the nonverbal immediacy measure. This leads to the production of a series of guidelines for social robot behaviour. The resulting behaviour is evaluated in a more general context, where both children and adults judge the immediacy of humans and robots in a similar manner, and their recall of a short story is tested. Children recall more of the story when the robot is more immediate, which demonstrates an e?ffect predicted by the literature. This study provides validation for the application of nonverbal immediacy to child-robot interaction. It is proposed that nonverbal immediacy measures could be used as a means of characterising robot social behaviour for human-robot interaction.

    @article{lirolem24215,
              volume = {9},
              number = {1},
               month = {January},
              author = {James Kennedy and Paul Baxter and Tony Belpaeme},
               title = {Nonverbal immediacy as a characterisation of social behaviour for human-robot interaction},
           publisher = {Springer},
                year = {2017},
             journal = {International Journal of Social Robotics},
               pages = {109--128},
            keywords = {ARRAY(0x7f78592da060)},
                 url = {http://eprints.lincoln.ac.uk/24215/},
            abstract = {An increasing amount of research has started
    to explore the impact of robot social behaviour on the
    outcome of a goal for a human interaction partner, such
    as cognitive learning gains. However, it remains unclear
    from what principles the social behaviour for such robots
    should be derived. Human models are often used, but
    in this paper an alternative approach is proposed. First,
    the concept of nonverbal immediacy from the communication
    literature is introduced, with a focus on how it
    can provide a characterisation of social behaviour, and
    the subsequent outcomes of such behaviour. A literature
    review is conducted to explore the impact on learning
    of the social cues which form the nonverbal immediacy
    measure. This leads to the production of a series
    of guidelines for social robot behaviour. The resulting
    behaviour is evaluated in a more general context, where
    both children and adults judge the immediacy of humans
    and robots in a similar manner, and their recall of
    a short story is tested. Children recall more of the story
    when the robot is more immediate, which demonstrates
    an e?ffect predicted by the literature. This study provides
    validation for the application of nonverbal immediacy
    to child-robot interaction. It is proposed that nonverbal
    immediacy measures could be used as a means of
    characterising robot social behaviour for human-robot
    interaction.}
    }
  • J. Kennedy, P. Baxter, and T. Belpaeme, “The impact of robot tutor nonverbal social behavior on child learning,” Frontiers in ICT, 2017.
    [BibTeX] [Abstract] [EPrints]

    Several studies have indicated that interacting with social robots in educational contexts may lead to a greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human?robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behavior a robot should employ in such interactions. Inspiration can be taken from human?human studies; this often leads to an assumption that the more social behavior an agent utilizes, the better the learning outcome will be. We apply a nonverbal behavior metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioral manipulations. We find a trend, which generally agrees with the pedagogy literature, but also that overt nonverbal behavior does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behavior and learning. We suggest that the combination of nonverbal behavior and social cue congruency is necessary to facilitate learning.

    @article{lirolem27043,
               month = {April},
               title = {The impact of robot tutor nonverbal social behavior on child learning},
              author = {James Kennedy and Paul Baxter and Tony Belpaeme},
           publisher = {Frontiers Media},
                year = {2017},
                note = {THIS ARTICLE IS PART OF THE RESEARCH TOPIC
    Affective and Social Signals for HRI},
             journal = {Frontiers in ICT},
            keywords = {ARRAY(0x7f78592ec118)},
                 url = {http://eprints.lincoln.ac.uk/27043/},
            abstract = {Several studies have indicated that interacting with social robots in educational contexts may lead to a greater learning than interactions with computers or virtual agents. As such, an increasing amount of social human?robot interaction research is being conducted in the learning domain, particularly with children. However, it is unclear precisely what social behavior a robot should employ in such interactions. Inspiration can be taken from human?human studies; this often leads to an assumption that the more social behavior an agent utilizes, the better the learning outcome will be. We apply a nonverbal behavior metric to a series of studies in which children are taught how to identify prime numbers by a robot with various behavioral manipulations. We find a trend, which generally agrees with the pedagogy literature, but also that overt nonverbal behavior does not account for all learning differences. We discuss the impact of novelty, child expectations, and responses to social cues to further the understanding of the relationship between robot social behavior and learning. We suggest that the combination of nonverbal behavior and social cue congruency is necessary to facilitate learning.}
    }
  • T. Krajnik, P. Cristoforis, K. Kusumam, P. Neubert, and T. Duckett, “Image features for visual teach-and-repeat navigation in changing environments,” Robotics and Autonomous Systems, vol. 88, pp. 127-141, 2017.
    [BibTeX] [Abstract] [EPrints]

    We present an evaluation of standard image features in the context of long-term visual teach-and-repeat navigation of mobile robots, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that for long-term autonomous navigation, the viewpoint-, scale- and rotation- invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We combine detection and description components of different image extractors and evaluate their performance on five datasets collected by mobile vehicles in three different outdoor environments over the course of one year. Moreover, we propose a trainable feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the most promising results were achieved by the SpG/CNN and the STAR/GRIEF feature, which was slightly less robust, but faster to calculate.

    @article{lirolem25239,
              volume = {88},
               month = {February},
              author = {Tomas Krajnik and Pablo Cristoforis and Keerthy Kusumam and Peer Neubert and Tom Duckett},
               title = {Image features for visual teach-and-repeat navigation in changing environments},
           publisher = {Elsevier},
             journal = {Robotics and Autonomous Systems},
               pages = {127--141},
                year = {2017},
            keywords = {ARRAY(0x7f78593edaf8)},
                 url = {http://eprints.lincoln.ac.uk/25239/},
            abstract = {We present an evaluation of standard image features in the context of long-term visual teach-and-repeat navigation of mobile robots, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that for long-term autonomous navigation, the viewpoint-, scale- and rotation- invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We combine detection and description components of different image extractors and evaluate their performance on five datasets collected by mobile vehicles in three different outdoor environments over the course of one year. Moreover, we propose a trainable feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the most promising results were achieved by the SpG/CNN and the STAR/GRIEF feature, which was slightly less robust, but faster to calculate.}
    }
  • T. Krajnik, J. P. Fentanes, J. Santos, and T. Duckett, “FreMEn: Frequency map enhancement for long-term mobile robot autonomy in changing environments,” Robotics, IEEE Transactions on [see also Robotics and Automation, IEEE Transactions on], 2017.
    [BibTeX] [Abstract] [EPrints]

    We present a new approach to long-term mobile robot mapping in dynamic indoor environments. Unlike traditional world models that are tailored to represent static scenes, our approach explicitly models environmental dynamics. We assume that some of the hidden processes that influence the dynamic environment states are periodic and model the uncertainty of the estimated state variables by their frequency spectra. The spectral model can represent arbitrary timescales of environment dynamics with low memory requirements. Transformation of the spectral model to the time domain allows for the prediction of the future environment states, which improves the robot’s long-term performance in dynamic environments. Experiments performed over time periods of months to years demonstrate that the approach can efficiently represent large numbers of observations and reliably predict future environment states. The experiments indicate that the model’s predictive capabilities improve mobile robot localisation and navigation in changing environments.

    @article{lirolem26196,
               month = {December},
               title = {FreMEn: Frequency map enhancement for long-term mobile robot autonomy in changing environments},
              author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Tom Duckett},
           publisher = {IEEE},
                year = {2017},
             journal = {Robotics, IEEE Transactions on [see also Robotics and Automation, IEEE Transactions on]},
            keywords = {ARRAY(0x7f78592c3060)},
                 url = {http://eprints.lincoln.ac.uk/26196/},
            abstract = {We present a new approach to long-term mobile robot mapping in dynamic indoor environments. Unlike traditional world models that are tailored to represent static scenes, our approach explicitly models environmental dynamics. We assume that some of the hidden processes that influence the dynamic environment states are periodic and model the uncertainty of the estimated state variables by their frequency spectra. The spectral model can represent arbitrary timescales of environment dynamics with low memory requirements. Transformation of the spectral model to the time domain allows for the prediction of the future environment states, which improves the robot's long-term performance in dynamic environments. Experiments performed over time periods of months to years demonstrate that the approach can efficiently represent large numbers of observations and reliably predict future environment states. The experiments indicate that the model's predictive capabilities improve mobile robot localisation and navigation in changing environments.}
    }
  • A. Kupcsik, M. P. Deisenroth, J. Peters, A. P. Loh, P. Vadakkepat, and G. Neumann, “Model-based contextual policy search for data-efficient generalization of robot skills,” Artificial Intelligence, 2017.
    [BibTeX] [Abstract] [EPrints]

    In robotics, lower-level controllers are typically used to make the robot solve a specific task in a fixed context. For example, the lower-level controller can encode a hitting movement while the context defines the target coordinates to hit. However, in many learning problems the context may change between task executions. To adapt the policy to a new context, we utilize a hierarchical approach by learning an upper-level policy that generalizes the lower-level controllers to new contexts. A common approach to learn such upper-level policies is to use policy search. However, the majority of current contextual policy search approaches are model-free and require a high number of interactions with the robot and its environment. Model-based approaches are known to significantly reduce the amount of robot experiments, however, current model-based techniques cannot be applied straightforwardly to the problem of learning contextual upper-level policies. They rely on specific parametrizations of the policy and the reward function, which are often unrealistic in the contextual policy search formulation. In this paper, we propose a novel model-based contextual policy search algorithm that is able to generalize lower-level controllers, and is data-efficient. Our approach is based on learned probabilistic forward models and information theoretic policy search. Unlike current algorithms, our method does not require any assumption on the parametrization of the policy or the reward function. We show on complex simulated robotic tasks and in a real robot experiment that the proposed learning framework speeds up the learning process by up to two orders of magnitude in comparison to existing methods, while learning high quality policies.

    @article{lirolem25774,
               month = {December},
               title = {Model-based contextual policy search for data-efficient generalization of robot skills},
              author = {A. Kupcsik and M. P. Deisenroth and J. Peters and A. P. Loh and P. Vadakkepat and G. Neumann},
           publisher = {Elsevier},
                year = {2017},
             journal = {Artificial Intelligence},
            keywords = {ARRAY(0x7f78592ddb48)},
                 url = {http://eprints.lincoln.ac.uk/25774/},
            abstract = {In robotics, lower-level controllers are typically used to make the robot solve a specific task in a fixed context. For example, the lower-level controller can encode a hitting movement while the context defines the target coordinates to hit. However, in many learning problems the context may change between task executions. To adapt the policy to a new context, we utilize a hierarchical approach by learning an upper-level policy that generalizes the lower-level controllers to new contexts. A common approach to learn such upper-level policies is to use policy search. However, the majority of current contextual policy search approaches are model-free and require a high number of interactions with the robot and its environment. Model-based approaches are known to significantly reduce the amount of robot experiments, however, current model-based techniques cannot be applied straightforwardly to the problem of learning contextual upper-level policies. They rely on specific parametrizations of the policy and the reward function, which are often unrealistic in the contextual policy search formulation. In this paper, we propose a novel model-based contextual policy search algorithm that is able to generalize lower-level controllers, and is data-efficient. Our approach is based on learned probabilistic forward models and information theoretic policy search. Unlike current algorithms, our method does not require any assumption on the parametrization of the policy or the reward function. We show on complex simulated robotic tasks and in a real robot experiment that the proposed learning framework speeds up the learning process by up to two orders of magnitude in comparison to existing methods, while learning high quality policies.}
    }
  • K. Kusumam, T. Krajnik, S. Pearson, T. Duckett, and G. Cielniak, “3D-vision based detection, localization, and sizing of broccoli heads in the field,” Journal of Field Robotics, 2017.
    [BibTeX] [Abstract] [EPrints]

    This paper describes a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors, which was developed and evaluated using sensory data collected under real-world field conditions in both the UK and Spain. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning, and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier, and a temporal filter to track the detected heads results in a system that detects broccoli heads with high precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field. Additionally, we present methods for automatically estimating the size of the broccoli heads, to determine when a head is ready for harvest. All of the methods were evaluated using ground-truth data from both the UK and Spain, which we also make available to the research community for subsequent algorithm development and result comparison. Cross-validation of the system trained on the UK dataset on the Spanish dataset, and vice versa, indicated good generalization capabilities of the system, confirming the strong potential of low-cost 3D imaging for commercial broccoli harvesting.

    @article{lirolem27782,
               month = {December},
               title = {3D-vision based detection, localization, and sizing of broccoli heads in the field},
              author = {Keerthy Kusumam and Tomas Krajnik and Simon Pearson and Tom Duckett and Grzegorz Cielniak},
           publisher = {Wiley Periodicals, Inc.},
                year = {2017},
             journal = {Journal of Field Robotics},
            keywords = {ARRAY(0x7f78592c2b68)},
                 url = {http://eprints.lincoln.ac.uk/27782/},
            abstract = {This paper describes a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors, which was developed and evaluated using sensory data collected under real-world field conditions in both the UK and Spain. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning, and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier, and a temporal filter to track the detected heads results in a system that detects broccoli heads with high precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field. Additionally, we present methods for automatically estimating the size of the broccoli heads, to determine when a head is ready for harvest. All of the methods were evaluated using ground-truth data from both the UK and Spain, which we also make available to the research community for subsequent algorithm development and result comparison. Cross-validation of the system trained on the UK dataset on the Spanish dataset, and vice versa, indicated good generalization capabilities of the system, confirming the strong potential of low-cost 3D imaging for commercial broccoli harvesting.}
    }
  • D. Liciotti, T. Duckett, N. Bellotto, E. Frontoni, and P. Zingaretti, “HMM-based activity recognition with a ceiling RGB-D camera,” in ICPRAM – 6th International Conference on Pattern Recognition Applications and Methods, 2017.
    [BibTeX] [Abstract] [EPrints]

    Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, the abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available.

    @inproceedings{lirolem25361,
           booktitle = {ICPRAM - 6th International Conference on Pattern Recognition Applications and Methods},
               month = {February},
               title = {HMM-based activity recognition with a ceiling RGB-D camera},
              author = {Daniele Liciotti and Tom Duckett and Nicola Bellotto and Emanuele Frontoni and Primo Zingaretti},
                year = {2017},
            keywords = {ARRAY(0x7f78593eda68)},
                 url = {http://eprints.lincoln.ac.uk/25361/},
            abstract = {Automated recognition of Activities of Daily Living allows to identify possible health problems and apply corrective strategies in Ambient Assisted Living (AAL). Activities of Daily Living analysis can provide very useful information for elder care and long-term care services. This paper presents an automated RGB-D video analysis system that recognises human ADLs activities, related to classical daily actions. The main goal is to predict the probability of an analysed subject action. Thus, the abnormal behaviour can be detected. The activity detection and recognition is performed using an affordable RGB-D camera. Human activities, despite their unstructured nature, tend to have a natural hierarchical structure; for instance, generally making a coffee involves a three-step process of turning on the coffee machine, putting sugar in cup and opening the fridge for milk. Action sequence recognition is then handled using a discriminative Hidden Markov Model (HMM). RADiaL, a dataset with RGB-D images and 3D position of each person for training as well as evaluating the HMM, has been built and made publicly available.}
    }
  • P. Lightbody, M. Hanheide, and T. Krajnik, “A versatile high-performance visual fiducial marker detection system with scalable identity encoding,” in 32nd ACM Symposium on Applied Computing, 2017, pp. 1-7.
    [BibTeX] [Abstract] [EPrints]

    Fiducial markers have a wide field of applications in robotics, ranging from external localisation of single robots or robotic swarms, over self-localisation in marker-augmented environments, to simplifying perception by tagging objects in a robot?s surrounding. We propose a new family of circular markers allowing for a computationally efficient detection, identification and full 3D position estimation. A key concept of our system is the separation of the detection and identification steps, where the first step is based on a computationally efficient circular marker detection, and the identification step is based on an open-ended ?Necklace code?, which allows for a theoretically infinite number of individually identifiable markers. The experimental evaluation of the system on a real robot indicates that while the proposed algorithm achieves similar accuracy to other state-of-the-art methods, it is faster by two orders of magnitude and it can detect markers from longer distances.

    @inproceedings{lirolem25828,
           booktitle = {32nd ACM Symposium on Applied Computing},
               month = {April},
               title = {A versatile high-performance visual fiducial marker detection system with scalable identity encoding},
              author = {Peter Lightbody and Marc Hanheide and Tomas Krajnik},
           publisher = {Association for Computing Machinery},
                year = {2017},
               pages = {1--7},
            keywords = {ARRAY(0x7f78592e44e0)},
                 url = {http://eprints.lincoln.ac.uk/25828/},
            abstract = {Fiducial markers have a wide field of applications in robotics, ranging from external localisation of single robots or robotic swarms, over self-localisation in marker-augmented environments, to simplifying perception by tagging objects in a robot?s surrounding. We propose a new family of circular markers allowing for a computationally efficient detection, identification and full 3D position estimation. A key concept of our system is the separation of the detection and identification steps, where the first step is based on a computationally efficient circular marker detection, and the identification step is based on an open-ended ?Necklace code?, which allows for a theoretically infinite number of individually identifiable markers. The experimental evaluation of the system on a real robot indicates that while the proposed algorithm achieves similar accuracy to other state-of-the-art methods, it is faster by two orders of magnitude and it can detect markers from longer distances.}
    }
  • R. Lioutikov, G. Neumann, G. Maeda, and J. Peters, “Learning movement primitive libraries through probabilistic segmentation,” International Journal of Robotics Research (IJRR), vol. 36, iss. 8, pp. 879-894, 2017.
    [BibTeX] [Abstract] [EPrints]

    Movement primitives are a well established approach for encoding and executing movements. While the primitives themselves have been extensively researched, the concept of movement primitive libraries has not received similar attention. Libraries of movement primitives represent the skill set of an agent. Primitives can be queried and sequenced in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into a representative set of primitives. Our proposed method differs from current approaches by taking advantage of the often neglected, mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. By exploiting this mutual dependency, we show that we can improve both the segmentation and the movement primitive library. Based on probabilistic inference our novel approach segments the demonstrations while learning a probabilistic representation of movement primitives. We demonstrate our method on two real robot applications. First, the robot segments sequences of different letters into a library, explaining the observed trajectories. Second, the robot segments demonstrations of a chair assembly task into a movement primitive library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.

    @article{lirolem28021,
              volume = {36},
              number = {8},
               month = {July},
              author = {Rudolf Lioutikov and Gerhard Neumann and Guilherme Maeda and Jan Peters},
               title = {Learning movement primitive libraries through probabilistic segmentation},
           publisher = {SAGE},
                year = {2017},
             journal = {International Journal of Robotics Research (IJRR)},
               pages = {879--894},
            keywords = {ARRAY(0x7f78592d9cb8)},
                 url = {http://eprints.lincoln.ac.uk/28021/},
            abstract = {Movement primitives are a well established approach for encoding and executing movements. While the primitives
    themselves have been extensively researched, the concept of movement primitive libraries has not received similar
    attention. Libraries of movement primitives represent the skill set of an agent. Primitives can be queried and sequenced
    in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into a representative
    set of primitives. Our proposed method differs from current approaches by taking advantage of the often neglected,
    mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. By
    exploiting this mutual dependency, we show that we can improve both the segmentation and the movement primitive
    library. Based on probabilistic inference our novel approach segments the demonstrations while learning a probabilistic
    representation of movement primitives. We demonstrate our method on two real robot applications. First, the robot
    segments sequences of different letters into a library, explaining the observed trajectories. Second, the robot segments
    demonstrations of a chair assembly task into a movement primitive library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.}
    }
  • D. Liu and S. Yue, “Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity,” Neurocomputing, 2017.
    [BibTeX] [Abstract] [EPrints]

    Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high e?ciency of the proposed method at an acceptable accuracy.

    @article{lirolem26922,
               month = {December},
               title = {Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity},
              author = {Daqi Liu and Shigang Yue},
           publisher = {Elsevier},
                year = {2017},
             journal = {Neurocomputing},
            keywords = {ARRAY(0x7f78592da078)},
                 url = {http://eprints.lincoln.ac.uk/26922/},
            abstract = {Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high e?ciency of the proposed method at an acceptable accuracy.}
    }
  • J. Lock, G. Cielniak, and N. Bellotto, “Portable navigations system with adaptive multimodal interface for the blind,” in AAAI 2017 Spring Symposium – Designing the User Experience of Machine Learning Systems, 2017.
    [BibTeX] [Abstract] [EPrints]

    Recent advances in mobile technology have the potential to radically change the quality of tools available for people with sensory impairments, in particular the blind. Nowadays almost every smart-phone and tablet is equipped with high resolutions cameras, which are typically used for photos and videos, communication purposes, games and virtual reality applications. Very little has been proposed to exploit these sensors for user localisation and navigation instead. To this end, the ?Active Vision with Human-in-the-Loop for the Visually Impaired? (ActiVis) project aims to develop a novel electronic travel aid to tackle the ?last 10 yards problem? and enable the autonomous navigation of blind users in unknown environments, ultimately enhancing or replacing existing solutions, such as guide dogs and white canes. This paper describes some of the key project?s challenges, in particular with respect to the design of the user interface that translate visual information from the camera to guiding instructions for the blind person, taking into account limitations due to the visual impairment and proposing a multimodal interface that embeds human-machine co-adaptation.

    @inproceedings{lirolem25413,
           booktitle = {AAAI 2017 Spring Symposium - Designing the User Experience of Machine Learning Systems},
               month = {March},
               title = {Portable navigations system with adaptive multimodal interface for the blind},
              author = {Jacobus Lock and Grzegorz Cielniak and Nicola Bellotto},
           publisher = {AAAI},
                year = {2017},
            keywords = {ARRAY(0x7f78592e0a20)},
                 url = {http://eprints.lincoln.ac.uk/25413/},
            abstract = {Recent advances in mobile technology have the potential to radically change the quality of tools available for people with sensory impairments, in particular the blind. Nowadays almost every smart-phone and tablet is equipped with high resolutions cameras, which are typically used for photos and videos, communication purposes, games and virtual reality applications. Very little has been proposed to exploit these sensors for user localisation and navigation instead. To this end, the ?Active Vision with Human-in-the-Loop for the Visually Impaired? (ActiVis) project aims to develop a novel electronic travel aid to tackle the ?last 10 yards problem? and enable the autonomous navigation of blind users in unknown environments, ultimately enhancing or replacing existing solutions, such as guide dogs and white canes. This paper describes some of the key project?s challenges, in particular with respect to the design of the user interface that translate visual information from the camera to guiding instructions for the blind person, taking into account limitations due to the visual impairment and proposing a multimodal interface that embeds human-machine co-adaptation.}
    }
  • G. Maeda, M. Ewerton, G. Neumann, R. Lioutikov, and J. Peters, “Phase estimation for fast action recognition and trajectory generation in human?robot collaboration,” The International Journal of Robotics Research, 2017.
    [BibTeX] [Abstract] [EPrints]

    This paper proposes a method to achieve fast and fluid human?robot interaction by estimating the progress of the movement of the human. The method allows the progress, also referred to as the phase of the movement, to be estimated even when observations of the human are partial and occluded; a problem typically found when using motion capture systems in cluttered environments. By leveraging on the framework of Interaction Probabilistic Movement Primitives, phase estimation makes it possible to classify the human action, and to generate a corresponding robot trajectory before the human finishes his/her movement. The method is therefore suited for semi-autonomous robots acting as assistants and coworkers. Since observations may be sparse, our method is based on computing the probability of different phase candidates to find the phase that best aligns the Interaction Probabilistic Movement Primitives with the current observations. The method is fundamentally different from approaches based on Dynamic Time Warping that must rely on a consistent stream of measurements at runtime. The resulting framework can achieve phase estimation, action recognition and robot trajectory coordination using a single probabilistic representation. We evaluated the method using a seven-degree-of-freedom lightweight robot arm equipped with a five-finger hand in single and multi-task collaborative experiments. We compare the accuracy achieved by phase estimation with our previous method based on dynamic time warping.

    @article{lirolem26734,
               month = {December},
               title = {Phase estimation for fast action recognition and trajectory generation in human?robot collaboration},
              author = {Guilherme Maeda and Marco Ewerton and Gerhard Neumann and Rudolf Lioutikov and Jan Peters},
           publisher = {SAGE},
                year = {2017},
             journal = {The International Journal of Robotics Research},
            keywords = {ARRAY(0x7f78592da030)},
                 url = {http://eprints.lincoln.ac.uk/26734/},
            abstract = {This paper proposes a method to achieve fast and fluid human?robot interaction by estimating the progress of the movement of the human. The method allows the progress, also referred to as the phase of the movement, to be estimated even when observations of the human are partial and occluded; a problem typically found when using motion capture systems in cluttered environments. By leveraging on the framework of Interaction Probabilistic Movement Primitives, phase estimation makes it possible to classify the human action, and to generate a corresponding robot trajectory before the human finishes his/her movement. The method is therefore suited for semi-autonomous robots acting as assistants and coworkers. Since observations may be sparse, our method is based on computing the probability of different phase candidates to find the phase that best aligns the Interaction Probabilistic Movement Primitives with the current observations. The method is fundamentally different from approaches based on Dynamic Time Warping that must rely on a consistent stream of measurements at runtime. The resulting framework can achieve phase estimation, action recognition and robot trajectory coordination using a single probabilistic representation. We evaluated the method using a seven-degree-of-freedom lightweight robot arm equipped with a five-finger hand in single and multi-task collaborative experiments. We compare the accuracy achieved by phase estimation with our previous method based on dynamic time warping.}
    }
  • G. J. Maeda, G. Neumann, M. Ewerton, R. Lioutikov, O. Kroemer, and J. Peters, “Probabilistic movement primitives for coordination of multiple human?robot collaborative tasks,” Autonomous Robots, vol. 41, iss. 3, pp. 593-612, 2017.
    [BibTeX] [Abstract] [EPrints]

    This paper proposes an interaction learning method for collaborative and assistive robots based on movement primitives. The method allows for both action recognition and human?robot movement coordination. It uses imitation learning to construct a mixture model of human?robot interaction primitives. This probabilistic model allows the assistive trajectory of the robot to be inferred from human observations. The method is scalable in relation to the number of tasks and can learn nonlinear correlations between the trajectories that describe the human?robot interaction. We evaluated the method experimentally with a lightweight robot arm in a variety of assistive scenarios, including the coordinated handover of a bottle to a human, and the collaborative assembly of a toolbox. Potential applications of the method are personal caregiver robots, control of intelligent prosthetic devices, and robot coworkers in factories.

    @article{lirolem25744,
              volume = {41},
              number = {3},
               month = {March},
              author = {G. J. Maeda and G. Neumann and M. Ewerton and R. Lioutikov and O. Kroemer and J. Peters},
                note = {Special Issue on Assistive and Rehabilitation Robotics},
               title = {Probabilistic movement primitives for coordination of multiple human?robot collaborative tasks},
           publisher = {Springer},
                year = {2017},
             journal = {Autonomous Robots},
               pages = {593--612},
            keywords = {ARRAY(0x7f78592b7b90)},
                 url = {http://eprints.lincoln.ac.uk/25744/},
            abstract = {This paper proposes an interaction learning method for collaborative and assistive robots based on movement primitives. The method allows for both action recognition and human?robot movement coordination. It uses imitation learning to construct a mixture model of human?robot interaction primitives. This probabilistic model allows the assistive trajectory of the robot to be inferred from human observations. The method is scalable in relation to the number of tasks and can learn nonlinear correlations between the trajectories that describe the human?robot interaction. We evaluated the method experimentally with a lightweight robot arm in a variety of assistive scenarios, including the coordinated handover of a bottle to a human, and the collaborative assembly of a toolbox. Potential applications of the method are personal caregiver robots, control of intelligent prosthetic devices, and robot coworkers in factories.}
    }
  • M. Mangan, S. Schwarz, B. Webb, A. Wystach, and J. Zeil, “How ants use vision when homing backward,” Current Biology, vol. 27, iss. 3, pp. 401-407, 2017.
    [BibTeX] [Abstract] [EPrints]

    Ants can navigate over long distances between their nest and food sites using visual cues [1 and 2]. Recent studies show that this capacity is undiminished when walking backward while dragging a heavy food item [3, 4 and 5]. This challenges the idea that ants use egocentric visual memories of the scene for guidance [1, 2 and 6]. Can ants use their visual memories of the terrestrial cues when going backward? Our results suggest that ants do not adjust their direction of travel based on the perceived scene while going backward. Instead, they maintain a straight direction using their celestial compass. This direction can be dictated by their path integrator [5] but can also be set using terrestrial visual cues after a forward peek. If the food item is too heavy to enable body rotations, ants moving backward drop their food on occasion, rotate and walk a few steps forward, return to the food, and drag it backward in a now-corrected direction defined by terrestrial cues. Furthermore, we show that ants can maintain their direction of travel independently of their body orientation. It thus appears that egocentric retinal alignment is required for visual scene recognition, but ants can translate this acquired directional information into a holonomic frame of reference, which enables them to decouple their travel direction from their body orientation and hence navigate backward. This reveals substantial flexibility and communication between different types of navigational information: from terrestrial to celestial cues and from egocentric to holonomic directional memories.

    @article{lirolem25891,
              volume = {27},
              number = {3},
               month = {February},
              author = {Michael Mangan and Sebastian Schwarz and Barbara Webb and Antoine Wystach and Jochen Zeil},
               title = {How ants use vision when homing backward},
           publisher = {Cell},
                year = {2017},
             journal = {Current Biology},
               pages = {401--407},
            keywords = {ARRAY(0x7f78593edb40)},
                 url = {http://eprints.lincoln.ac.uk/25891/},
            abstract = {Ants can navigate over long distances between their nest and food sites using visual cues [1 and 2]. Recent studies show that this capacity is undiminished when walking backward while dragging a heavy food item [3, 4 and 5]. This challenges the idea that ants use egocentric visual memories of the scene for guidance [1, 2 and 6]. Can ants use their visual memories of the terrestrial cues when going backward? Our results suggest that ants do not adjust their direction of travel based on the perceived scene while going backward. Instead, they maintain a straight direction using their celestial compass. This direction can be dictated by their path integrator [5] but can also be set using terrestrial visual cues after a forward peek. If the food item is too heavy to enable body rotations, ants moving backward drop their food on occasion, rotate and walk a few steps forward, return to the food, and drag it backward in a now-corrected direction defined by terrestrial cues. Furthermore, we show that ants can maintain their direction of travel independently of their body orientation. It thus appears that egocentric retinal alignment is required for visual scene recognition, but ants can translate this acquired directional information into a holonomic frame of reference, which enables them to decouple their travel direction from their body orientation and hence navigate backward. This reveals substantial flexibility and communication between different types of navigational information: from terrestrial to celestial cues and from egocentric to holonomic directional memories.}
    }
  • O. M. Mozos, V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, R. Dobrescu, and J. M. Ferrandez, “Stress detection using wearable physiological and sociometric sensors,” International Journal of Neural Systems, vol. 27, iss. 2, p. 1650041, 2017.
    [BibTeX] [Abstract] [EPrints]

    Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbour. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real time stress detection. Finally, we present an study of the most discriminative features for stress detection.

    @article{lirolem23128,
              volume = {27},
              number = {2},
               month = {March},
              author = {Oscar Martinez Mozos and Virginia Sandulescu and Sally Andrews and David Ellis and Nicola Bellotto and Radu Dobrescu and Jose Manuel Ferrandez},
               title = {Stress detection using wearable physiological and sociometric sensors},
           publisher = {World Scientific Publishing},
                year = {2017},
             journal = {International Journal of Neural Systems},
               pages = {1650041},
            keywords = {ARRAY(0x7f78593edab0)},
                 url = {http://eprints.lincoln.ac.uk/23128/},
            abstract = {Stress remains a significant social problem for individuals in modern societies. This paper presents a machine learning approach for the automatic detection of stress of people in a social situation by combining two sensor systems that capture physiological and social responses. We compare the performance using different classifiers including support vector machine, AdaBoost, and k-nearest neighbour. Our experimental results show that by combining the measurements from both sensor systems, we could accurately discriminate between stressful and neutral situations during a controlled Trier social stress test (TSST). Moreover, this paper assesses the discriminative ability of each sensor modality individually and considers their suitability for real time stress detection. Finally, we present an study of the most discriminative features for stress detection.}
    }
  • T. Osa, A. G. M. Esfahani, R. Stolkin, R. Lioutikov, J. Peters, and G. Neumann, “Guiding trajectory optimization by demonstrated distributions,” IEEE Robotics and Automation Letters (RA-L), vol. 2, iss. 2, pp. 819-826, 2017.
    [BibTeX] [Abstract] [EPrints]

    Trajectory optimization is an essential tool for motion planning under multiple constraints of robotic manipulators. Optimization-based methods can explicitly optimize a trajectory by leveraging prior knowledge of the system and have been used in various applications such as collision avoidance. However, these methods often require a hand-coded cost function in order to achieve the desired behavior. Specifying such cost function for a complex desired behavior, e.g., disentangling a rope, is a nontrivial task that is often even infeasible. Learning from demonstration (LfD) methods offer an alternative way to program robot motion. LfD methods are less dependent on analytical models and instead learn the behavior of experts implicitly from the demonstrated trajectories. However, the problem of adapting the demonstrations to new situations, e.g., avoiding newly introduced obstacles, has not been fully investigated in the literature. In this paper, we present a motion planning framework that combines the advantages of optimization-based and demonstration-based methods. We learn a distribution of trajectories demonstrated by human experts and use it to guide the trajectory optimization process. The resulting trajectory maintains the demonstrated behaviors, which are essential to performing the task successfully, while adapting the trajectory to avoid obstacles. In simulated experiments and with a real robotic system, we verify that our approach optimizes the trajectory to avoid obstacles and encodes the demonstrated behavior in the resulting trajectory

    @article{lirolem26731,
              volume = {2},
              number = {2},
               month = {January},
              author = {Takayuki Osa and Amir M. Ghalamzan Esfahani and Rustam Stolkin and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
               title = {Guiding trajectory optimization by demonstrated distributions},
           publisher = {IEEE},
                year = {2017},
             journal = {IEEE Robotics and Automation Letters (RA-L)},
               pages = {819--826},
            keywords = {ARRAY(0x7f78592c3048)},
                 url = {http://eprints.lincoln.ac.uk/26731/},
            abstract = {Trajectory optimization is an essential tool for motion
    planning under multiple constraints of robotic manipulators.
    Optimization-based methods can explicitly optimize a trajectory
    by leveraging prior knowledge of the system and have been used
    in various applications such as collision avoidance. However, these
    methods often require a hand-coded cost function in order to
    achieve the desired behavior. Specifying such cost function for
    a complex desired behavior, e.g., disentangling a rope, is a nontrivial
    task that is often even infeasible. Learning from demonstration
    (LfD) methods offer an alternative way to program robot
    motion. LfD methods are less dependent on analytical models
    and instead learn the behavior of experts implicitly from the
    demonstrated trajectories. However, the problem of adapting the
    demonstrations to new situations, e.g., avoiding newly introduced
    obstacles, has not been fully investigated in the literature. In this
    paper, we present a motion planning framework that combines
    the advantages of optimization-based and demonstration-based
    methods. We learn a distribution of trajectories demonstrated by
    human experts and use it to guide the trajectory optimization
    process. The resulting trajectory maintains the demonstrated
    behaviors, which are essential to performing the task successfully,
    while adapting the trajectory to avoid obstacles. In simulated
    experiments and with a real robotic system, we verify that our
    approach optimizes the trajectory to avoid obstacles and encodes
    the demonstrated behavior in the resulting trajectory}
    }
  • J. Pajarinen, V. Kyrki, M. Koval, S. Srinivasa, J. Peters, and G. Neumann, “Hybrid control trajectory optimization under uncertainty,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017.
    [BibTeX] [Abstract] [EPrints]

    Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.

    @inproceedings{lirolem28257,
           booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
               month = {September},
               title = {Hybrid control trajectory optimization under uncertainty},
              author = {J. Pajarinen and V. Kyrki and M. Koval and S Srinivasa and J. Peters and G. Neumann},
                year = {2017},
            keywords = {ARRAY(0x7f78592ee700)},
                 url = {http://eprints.lincoln.ac.uk/28257/},
            abstract = {Trajectory optimization is a fundamental problem in robotics. While optimization of continuous control trajectories is well developed, many applications require both discrete and continuous, i.e. hybrid controls. Finding an optimal sequence of hybrid controls is challenging due to the exponential explosion of discrete control combinations. Our method, based on Differential Dynamic Programming (DDP), circumvents this problem by incorporating discrete actions inside DDP: we first optimize continuous mixtures of discrete actions, and, subsequently force the mixtures into fully discrete actions. Moreover, we show how our approach can be extended to partially observable Markov decision processes (POMDPs) for trajectory planning under uncertainty. We validate the approach in a car driving problem where the robot has to switch discrete gears and in a box pushing application where the robot can switch the side of the box to push. The pose and the friction parameters of the pushed box are initially unknown and only indirectly observable.}
    }
  • A. Paraschos, R. Lioutikov, J. Peters, and G. Neumann, “Probabilistic prioritization of movement primitives,” IEEE Robotics and Automation Letters, vol. PP, iss. 99, 2017.
    [BibTeX] [Abstract] [EPrints]

    Movement prioritization is a common approach to combine controllers of different tasks for redundant robots, where each task is assigned a priority. The priorities of the tasks are often hand-tuned or the result of an optimization, but seldomly learned from data. This paper combines Bayesian task prioritization with probabilistic movement primitives to prioritize full motion sequences that are learned from demonstrations. Probabilistic movement primitives (ProMPs) can encode distributions of movements over full motion sequences and provide control laws to exactly follow these distributions. The probabilistic formulation allows for a natural application of Bayesian task prioritization. We extend the ProMP controllers with an additional feedback component that accounts inaccuracies in following the distribution and allows for a more robust prioritization of primitives. We demonstrate how the task priorities can be obtained from imitation learning and how different primitives can be combined to solve even unseen task-combinations. Due to the prioritization, our approach can efficiently learn a combination of tasks without requiring individual models per task combination. Further, our approach can adapt an existing primitive library by prioritizing additional controllers, for example, for implementing obstacle avoidance. Hence, the need of retraining the whole library is avoided in many cases. We evaluate our approach on reaching movements under constraints with redundant simulated planar robots and two physical robot platforms, the humanoid robot ?iCub? and a KUKA LWR robot arm.

    @article{lirolem27901,
              volume = {PP},
              number = {99},
               month = {July},
              author = {Alexandros Paraschos and Rudolf Lioutikov and Jan Peters and Gerhard Neumann},
           booktitle = {, Proceedings of the International Conference on Intelligent Robot Systems, and IEEE Robotics and Automation Letters (RA-L)},
               title = {Probabilistic prioritization of movement primitives},
           publisher = {IEEE},
                year = {2017},
             journal = {IEEE Robotics and Automation Letters},
            keywords = {ARRAY(0x7f78592c6158)},
                 url = {http://eprints.lincoln.ac.uk/27901/},
            abstract = {Movement prioritization is a common approach
    to combine controllers of different tasks for redundant robots,
    where each task is assigned a priority. The priorities of the
    tasks are often hand-tuned or the result of an optimization,
    but seldomly learned from data. This paper combines Bayesian
    task prioritization with probabilistic movement primitives to
    prioritize full motion sequences that are learned from demonstrations.
    Probabilistic movement primitives (ProMPs) can
    encode distributions of movements over full motion sequences
    and provide control laws to exactly follow these distributions.
    The probabilistic formulation allows for a natural application of
    Bayesian task prioritization. We extend the ProMP controllers
    with an additional feedback component that accounts inaccuracies
    in following the distribution and allows for a more
    robust prioritization of primitives. We demonstrate how the
    task priorities can be obtained from imitation learning and
    how different primitives can be combined to solve even unseen
    task-combinations. Due to the prioritization, our approach can
    efficiently learn a combination of tasks without requiring individual
    models per task combination. Further, our approach can
    adapt an existing primitive library by prioritizing additional
    controllers, for example, for implementing obstacle avoidance.
    Hence, the need of retraining the whole library is avoided in
    many cases. We evaluate our approach on reaching movements
    under constraints with redundant simulated planar robots and
    two physical robot platforms, the humanoid robot ?iCub? and
    a KUKA LWR robot arm.}
    }
  • A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Using probabilistic movement primitives in robotics,” Autonomous Robots, 2017.
    [BibTeX] [Abstract] [EPrints]

    Movement Primitives are a well-established paradigm for modular movement representation and generation. They provide a data-driven representation of movements and support generalization to novel situations, temporal modulation, sequencing of primitives and controllers for executing the primitive on physical systems. However, while many MP frameworks exhibit some of these properties, there is a need for a uni- fied framework that implements all of them in a principled way. In this paper, we show that this goal can be achieved by using a probabilistic representation. Our approach models trajectory distributions learned from stochastic movements. Probabilistic operations, such as conditioning can be used to achieve generalization to novel situations or to combine and blend movements in a principled way. We derive a stochastic feedback controller that reproduces the encoded variability of the movement and the coupling of the degrees of freedom of the robot. We evaluate and compare our approach on several simulated and real robot scenarios.

    @article{lirolem27883,
               month = {December},
               title = {Using probabilistic movement primitives in robotics},
              author = {Alexandros Paraschos and Christian Daniel and Jan Peters and Gerhard Neumann},
           publisher = {Springer Verlag},
                year = {2017},
             journal = {Autonomous Robots},
            keywords = {ARRAY(0x7f78592c2958)},
                 url = {http://eprints.lincoln.ac.uk/27883/},
            abstract = {Movement Primitives are a well-established
    paradigm for modular movement representation and
    generation. They provide a data-driven representation
    of movements and support generalization to novel situations,
    temporal modulation, sequencing of primitives
    and controllers for executing the primitive on physical
    systems. However, while many MP frameworks exhibit
    some of these properties, there is a need for a uni-
    fied framework that implements all of them in a principled
    way. In this paper, we show that this goal can be
    achieved by using a probabilistic representation. Our
    approach models trajectory distributions learned from
    stochastic movements. Probabilistic operations, such as
    conditioning can be used to achieve generalization to
    novel situations or to combine and blend movements in
    a principled way. We derive a stochastic feedback controller
    that reproduces the encoded variability of the
    movement and the coupling of the degrees of freedom
    of the robot. We evaluate and compare our approach
    on several simulated and real robot scenarios.}
    }
  • A. Rahman, A. Ahmed, and S. Yue, “Classification of tongue – glossitis abnormality,” in International Conference of Data Mining and Knowledge Engineering, 2017, pp. 1-4.
    [BibTeX] [Abstract] [EPrints]

    An approach to classify tongue abnormality related to Diabetes Mellitus (DM) following Western Medicine (WM) approach. Glossitis abnormality is one of the common tongue abnormalities that affects patients who suffer from Diabetes Mellitus (DM). The novelty of the proposed approach is attributed to utilising visual signs that appear on tongue due to Glossitis abnormality causes by high blood sugar level in the human body. The test for the blood sugar level is inconvenient for some patients in rural and poor areas where medical services are minimal or may not be available at all. To screen and monitor human organ effectively, the proposed computer aided model predicts and classifies abnormality appears on the tongue or tongue surface using visual signs caused by the abnormality. The visual signs were extracted following a logically formed medical approach, which complies with Western Medicine (WM) approach. Using Random Forest classifier on the extracted visual tongue signs, from 572 tongue samples for 166 patients, the experimental results have shown promising accuracy of 95.8\% for Glossitis abnormality.

    @inproceedings{lirolem27400,
           booktitle = {International Conference of Data Mining and Knowledge Engineering},
               month = {July},
               title = {Classification of tongue - glossitis abnormality},
              author = {Ashiqur Rahman and Amr Ahmed and Shigang Yue},
                year = {2017},
               pages = {1--4},
            keywords = {ARRAY(0x7f78592e1f88)},
                 url = {http://eprints.lincoln.ac.uk/27400/},
            abstract = {An approach to classify tongue abnormality related to Diabetes Mellitus (DM) following Western Medicine (WM) approach. Glossitis abnormality is one of the common tongue abnormalities that affects patients who suffer from Diabetes Mellitus (DM).
    
    The novelty of the proposed approach is attributed to utilising visual signs that appear on tongue due to Glossitis abnormality causes by high blood sugar level in the human body. The test for the blood sugar level is inconvenient for some patients in rural and poor areas where medical services are minimal or may not be available at all. To screen and monitor human organ effectively, the proposed computer aided model predicts and classifies abnormality appears on the tongue or tongue surface using visual signs caused by the abnormality. The visual signs were extracted following a logically formed medical approach, which complies with Western Medicine (WM) approach. Using Random Forest classifier on the extracted visual tongue signs, from 572 tongue samples for 166 patients, the experimental results have shown promising accuracy of 95.8\% for Glossitis abnormality.}
    }
  • M. Salem, A. Weiss, and P. Baxter, “New frontiers in human-robot interaction [special section on interdisciplinary human-centred approaches],” Interaction Studies, vol. 17, iss. 3, pp. 405-407, 2017.
    [BibTeX] [Abstract] [EPrints]

    @article{lirolem27044,
              volume = {17},
              number = {3},
               month = {March},
              author = {Maha Salem and Astrid Weiss and Paul Baxter},
               title = {New frontiers in human-robot interaction [special section on interdisciplinary human-centred approaches]},
           publisher = {John Benjamins Publishers},
                year = {2017},
             journal = {Interaction Studies},
               pages = {405--407},
            keywords = {ARRAY(0x7f78592ec4a8)},
                 url = {http://eprints.lincoln.ac.uk/27044/},
            abstract = {-}
    }
  • J. M. Santos, T. Krajník, and T. Duckett, “Spatio-temporal exploration strategies for long-term autonomy of mobile robots,” Robotics and Autonomous Systems, vol. 88, pp. 116-126, 2017.
    [BibTeX] [Abstract] [EPrints]

    We present a study of spatio-temporal environment representations and exploration strategies for long-term deployment of mobile robots in real-world, dynamic environments. We propose a new concept for life-long mobile robot spatio-temporal exploration that aims at building, updating and maintaining the environment model during the long-term deployment. The addition of the temporal dimension to the explored space makes the exploration task a never-ending data-gathering process, which we address by application of information-theoretic exploration techniques to world representations that model the uncertainty of environment states as probabilistic functions of time. We evaluate the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months. The combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of continuously changing environments, enabling efficient and self-improving long-term operation of mobile robots.

    @article{lirolem25412,
              volume = {88},
               month = {February},
              author = {Jo{\~a}o Machado Santos and Tom{\'a}{\vs} Krajn{\'i}k and Tom Duckett},
               title = {Spatio-temporal exploration strategies for long-term autonomy of mobile robots},
           publisher = {Elsevier},
             journal = {Robotics and Autonomous Systems},
               pages = {116--126},
                year = {2017},
            keywords = {ARRAY(0x7f78593edbd0)},
                 url = {http://eprints.lincoln.ac.uk/25412/},
            abstract = {We present a study of spatio-temporal environment representations and exploration strategies for long-term deployment of mobile robots in real-world, dynamic environments.
    We propose a new concept for life-long mobile robot spatio-temporal exploration that aims at building, updating and maintaining the environment model during the long-term deployment.
    The addition of the temporal dimension to the explored space makes the exploration task a never-ending data-gathering process, which we address by application of information-theoretic exploration techniques to world representations that model the uncertainty of environment states as probabilistic functions of time.
    We evaluate the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months.
    The combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of continuously changing environments, enabling efficient and self-improving long-term operation of mobile robots.}
    }
  • E. Senft, P. Baxter, J. Kennedy, S. Lemaignan, and T. Belpaeme, “Supervised autonomy for online learning in human-robot interaction,” Pattern Recognition Letters, 2017.
    [BibTeX] [Abstract] [EPrints]

    When a robot is learning it needs to explore its environment and how its environment responds on its actions. When the environment is large and there are a large number of possible actions the robot can take, this exploration phase can take prohibitively long. However, exploration can often be optimised by letting a human expert guide the robot during its learning. Interactive machine learning, in which a human user interactively guides the robot as it learns, has been shown to be an effective way to teach a robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on the robot?s progress. This paper presents a novel method which combines Reinforcement Learning and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy while maintaining human supervisory oversight of the robot?s behaviour. This method is evaluated and compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high workload on the human teacher.

    @article{lirolem26857,
               month = {December},
               title = {Supervised autonomy for online learning in human-robot interaction},
              author = {Emmanuel Senft and Paul Baxter and James Kennedy and Severin Lemaignan and Tony Belpaeme},
           publisher = {Elsevier / North Holland for International Association for Pattern Recognition},
                year = {2017},
             journal = {Pattern Recognition Letters},
            keywords = {ARRAY(0x7f78592e6480)},
                 url = {http://eprints.lincoln.ac.uk/26857/},
            abstract = {When a robot is learning it needs to explore its environment and how its environment responds on its
    actions. When the environment is large and there are a large number of possible actions the robot can
    take, this exploration phase can take prohibitively long. However, exploration can often be optimised
    by letting a human expert guide the robot during its learning. Interactive machine learning, in which a
    human user interactively guides the robot as it learns, has been shown to be an effective way to teach a
    robot. It requires an intuitive control mechanism to allow the human expert to provide feedback on
    the robot?s progress. This paper presents a novel method which combines Reinforcement Learning
    and Supervised Progressively Autonomous Robot Competencies (SPARC). By allowing the user to
    fully control the robot and by treating rewards as implicit, SPARC aims to learn an action policy
    while maintaining human supervisory oversight of the robot?s behaviour. This method is evaluated and
    compared to Interactive Reinforcement Learning in a robot teaching task. Qualitative and quantitative
    results indicate that SPARC allows for safer and faster learning by the robot, whilst not placing a high
    workload on the human teacher.}
    }
  • V. Tangkaratt, H. van Hoof, S. Parisi, G. Neumann, J. Peters, and M. Sugiyama, “Policy search with high-dimensional context variables,” in AAAI Conference on Artificial Intelligence (AAAI), 2017.
    [BibTeX] [Abstract] [EPrints]

    Direct contextual policy search methods learn to improve policy parameters and simultaneously generalize these parameters to different context or task variables. However, learning from high-dimensional context variables, such as camera images, is still a prominent problem in many real-world tasks. A naive application of unsupervised dimensionality reduction methods to the context variables, such as principal component analysis, is insufficient as task-relevant input may be ignored. In this paper, we propose a contextual policy search method in the model-based relative entropy stochastic search framework with integrated dimensionality reduction. We learn a model of the reward that is locally quadratic in both the policy parameters and the context variables. Furthermore, we perform supervised linear dimensionality reduction on the context variables by nuclear norm regularization. The experimental results show that the proposed method outperforms naive dimensionality reduction via principal component analysis and a state-of-the-art contextual policy search method.

    @inproceedings{lirolem26740,
           booktitle = {AAAI Conference on Artificial Intelligence (AAAI)},
               month = {February},
               title = {Policy search with high-dimensional context variables},
              author = {V. Tangkaratt and H. van Hoof and S. Parisi and G. Neumann and J. Peters and M. Sugiyama},
           publisher = {Association for the Advancement of Artificial Intelligence},
                year = {2017},
            keywords = {ARRAY(0x7f78593edc00)},
                 url = {http://eprints.lincoln.ac.uk/26740/},
            abstract = {Direct contextual policy search methods learn to improve policy
    parameters and simultaneously generalize these parameters
    to different context or task variables. However, learning
    from high-dimensional context variables, such as camera images,
    is still a prominent problem in many real-world tasks.
    A naive application of unsupervised dimensionality reduction
    methods to the context variables, such as principal component
    analysis, is insufficient as task-relevant input may be ignored.
    In this paper, we propose a contextual policy search method in
    the model-based relative entropy stochastic search framework
    with integrated dimensionality reduction. We learn a model of
    the reward that is locally quadratic in both the policy parameters
    and the context variables. Furthermore, we perform supervised
    linear dimensionality reduction on the context variables
    by nuclear norm regularization. The experimental results
    show that the proposed method outperforms naive dimensionality
    reduction via principal component analysis and
    a state-of-the-art contextual policy search method.}
    }
  • D. Wang, X. Hou, J. Xu, S. Yue, and C. Liu, “Traffic sign detection using a cascade method with fast feature extraction and saliency test,” IEEE Transactions on Intelligent Transportation Systems, vol. 99, pp. 1-13, 2017.
    [BibTeX] [Abstract] [EPrints]

    Automatic traffic sign detection is challenging due to the complexity of scene images, and fast detection is required in real applications such as driver assistance systems. In this paper, we propose a fast traffic sign detection method based on a cascade method with saliency test and neighboring scale awareness. In the cascade method, feature maps of several channels are extracted efficiently using approximation techniques. Sliding windows are pruned hierarchically using coarse-to-fine classifiers and the correlation between neighboring scales. The cascade system has only one free parameter, while the multiple thresholds are selected by a data-driven approach. To further increase speed, we also use a novel saliency test based on mid-level features to pre-prune background windows. Experiments on two public traffic sign data sets show that the proposed method achieves competing performance and runs 27 times as fast as most of the state-of-the-art methods.

    @article{lirolem27022,
              volume = {99},
               month = {April},
              author = {Dongdong Wang and Xinwen Hou and Jiawei Xu and Shigang Yue and Cheng-Lin Liu},
               title = {Traffic sign detection using a cascade method with fast feature extraction and saliency test},
           publisher = {IEEE},
             journal = {IEEE Transactions on Intelligent Transportation Systems},
               pages = {1--13},
                year = {2017},
            keywords = {ARRAY(0x7f78592c2b80)},
                 url = {http://eprints.lincoln.ac.uk/27022/},
            abstract = {Automatic traffic sign detection is challenging due to the complexity of scene images, and fast detection is required in real applications such as driver assistance systems. In this paper, we propose a fast traffic sign detection method based on a cascade method with saliency test and neighboring scale awareness. In the cascade method, feature maps of several channels are extracted efficiently using approximation techniques. Sliding windows are pruned hierarchically using coarse-to-fine classifiers and the correlation between neighboring scales. The cascade system has only one free parameter, while the multiple thresholds are selected by a data-driven approach. To further increase speed, we also use a novel saliency test based on mid-level features to pre-prune background windows. Experiments on two public traffic sign data sets show that the proposed method achieves competing performance and runs 27 times as fast as most of the state-of-the-art methods.}
    }
  • J. Xu, S. Yue, F. Menchinelli, and K. Guo, “What has been missed for predicting human attention in viewing driving clips?,” PeerJ, vol. 5, p. e2946, 2017.
    [BibTeX] [Abstract] [EPrints]

    Recent research progress on the topic of human visual attention allocation in scene perception and its simulation is based mainly on studies with static images. However, natural vision requires us to extract visual information that constantly changes due to egocentric movements or dynamics of the world. It is unclear to what extent spatio-temporal regularity, an inherent regularity in dynamic vision, affects human gaze distribution and saliency computation in visual attention models. In this free-viewing eye-tracking study we manipulated the spatio-temporal regularity of traffic videos by presenting them in normal video sequence, reversed video sequence, normal frame sequence, and randomised frame sequence. The recorded human gaze allocation was then used as the ?ground truth? to examine the predictive ability of a number of state-of-the-art visual attention models. The analysis revealed high inter-observer agreement across individual human observers, but all the tested attention models performed significantly worse than humans. The inferior predictability of the models was evident from indistinguishable gaze prediction irrespective of stimuli presentation sequence, and weak central fixation bias. Our findings suggest that a realistic visual attention model for the processing of dynamic scenes should incorporate human visual sensitivity with spatio-temporal regularity and central fixation bias.

    @article{lirolem25963,
              volume = {5},
               month = {February},
              author = {Jiawei Xu and Shigang Yue and Federica Menchinelli and Kun Guo},
               title = {What has been missed for predicting human attention in viewing driving clips?},
           publisher = {PeerJ},
             journal = {PeerJ},
               pages = {e2946},
                year = {2017},
            keywords = {ARRAY(0x7f78593edb88)},
                 url = {http://eprints.lincoln.ac.uk/25963/},
            abstract = {Recent research progress on the topic of human visual attention allocation in scene perception and its simulation is based mainly on studies with static images. However, natural vision requires us to extract visual information that constantly changes due to egocentric movements or dynamics of the world. It is unclear to what extent spatio-temporal regularity, an inherent regularity in dynamic vision, affects human gaze distribution and saliency computation in visual attention models. In this free-viewing eye-tracking study we manipulated the spatio-temporal regularity of traffic videos by presenting them in normal video sequence, reversed video sequence, normal frame sequence, and randomised frame sequence. The recorded human gaze allocation was then used as the ?ground truth? to examine the predictive ability of a number of state-of-the-art visual attention models. The analysis revealed high inter-observer agreement across individual human observers, but all the tested attention models performed significantly worse than humans. The inferior predictability of the models was evident from indistinguishable gaze prediction irrespective of stimuli presentation sequence, and weak central fixation bias. Our findings suggest that a realistic visual attention model for the processing of dynamic scenes should incorporate human visual sensitivity with spatio-temporal regularity and central fixation bias.}
    }
  • Z. Yan, T. Duckett, and N. Bellotto, “Online learning for human classification in 3D LiDAR-based tracking,” in IEEE/RSJ International Conference on Itelligent Robots and Systems (IROS), 2017.
    [BibTeX] [Abstract] [EPrints]

    Human detection and tracking is one of the most important aspects to be considered in service robotics, as the robot often shares its workspace and interacts closely with humans. This paper presents an online learning framework for human classification in 3D LiDAR scans, taking advantage of robust multi-target tracking to avoid the need for data annotation by a human expert. The system learns iteratively by retraining a classifier online with the samples collected by the robot over time. A novel aspect of our approach is that errors in training data can be corrected using the information provided by the 3D LiDAR-based tracking. In order to do this, an efficient 3D cluster detector of potential human targets has been implemented. We evaluate the framework using a new 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments analyse the real-time performance of the cluster detector and show that our online-trained human classifier matches and in some cases outperforms its offline version.

    @inproceedings{lirolem27675,
           booktitle = {IEEE/RSJ International Conference on Itelligent Robots and Systems (IROS)},
               month = {September},
               title = {Online learning for human classification in 3D LiDAR-based tracking},
              author = {Zhi Yan and Tom Duckett and Nicola Bellotto},
           publisher = {IEEE},
                year = {2017},
            keywords = {ARRAY(0x7f78592bfd78)},
                 url = {http://eprints.lincoln.ac.uk/27675/},
            abstract = {Human detection and tracking is one of the most important aspects to be considered in service robotics, as the robot often shares its workspace and interacts closely with humans. This paper presents an online learning framework for human classification in 3D LiDAR scans, taking advantage of robust multi-target tracking to avoid the need for data annotation by a human expert. The system learns iteratively by retraining a classifier online with the samples collected by the robot over time. A novel aspect of our approach is that errors in training data can be corrected using the information provided by the 3D LiDAR-based tracking. In order to do this, an efficient 3D cluster detector of potential human targets has been implemented. We evaluate the framework using a new 3D LiDAR dataset of people moving in a large indoor public space, which is made available to the research community. The experiments analyse the real-time performance of the cluster detector and show that our online-trained human classifier matches and in some cases outperforms its offline version.}
    }
  • A. Zaganidis, M. Magnusson, T. Duckett, and G. Cielniak, “Semantic-assisted 3D Normal Distributions Transform for scan registration in environments with limited structure,” in International Conference on Intelligent Robots and Systems (IROS), 2017.
    [BibTeX] [Abstract] [EPrints]

    Point cloud registration is a core problem of many robotic applications, including simultaneous localization and mapping. The Normal Distributions Transform (NDT) is a method that fits a number of Gaussian distributions to the data points, and then uses this transform as an approximation of the real data, registering a relatively small number of distributions as opposed to the full point cloud. This approach contributes to NDT?s registration robustness and speed but leaves room for improvement in environments of limited structure. To address this limitation we propose a method for the introduction of semantic information extracted from the point clouds into the registration process. The paper presents a large scale experimental evaluation of the algorithm against NDT on two publicly available benchmark data sets. For the purpose of this test a measure of smoothness is used for the semantic partitioning of the point clouds. The results indicate that the proposed method improves the accuracy, robustness and speed of NDT registration, especially in unstructured environments, making NDT suitable for a wider range of applications.

    @inproceedings{lirolem28481,
           booktitle = {International Conference on Intelligent Robots and Systems (IROS)},
               month = {September},
               title = {Semantic-assisted 3D Normal Distributions Transform for scan registration in environments with limited structure},
              author = {Anestis Zaganidis and Martin Magnusson and Tom Duckett and Grzegorz Cielniak},
           publisher = {IEEE/RSJ},
                year = {2017},
            keywords = {ARRAY(0x7f78592c2fa0)},
                 url = {http://eprints.lincoln.ac.uk/28481/},
            abstract = {Point cloud registration is a core problem of many robotic applications, including simultaneous localization and mapping. The Normal Distributions Transform (NDT) is a method that fits a number of Gaussian distributions to the data points, and then uses this transform as an approximation of the real data, registering a relatively small number of distributions as opposed to the full point cloud. This approach contributes to NDT?s registration robustness and speed but leaves room for improvement in environments of limited structure. 
    To address this limitation we propose a method for the introduction of semantic information extracted from the point clouds into the registration process. The paper presents a large scale experimental evaluation of the algorithm against NDT on two publicly available benchmark data sets. For the purpose of this test a measure of smoothness is used for the semantic partitioning of the point clouds. The results indicate that the proposed method improves the accuracy, robustness and speed of NDT registration, especially in unstructured environments, making NDT suitable for a wider range of applications.}
    }

2016

  • A. Abdolmaleki, R. Lioutikov, N. Lua, P. L. Reis, J. Peters, and G. Neumann, “Model-based relative entropy stochastic search,” in Advances in Neural Information Processing Systems (NIPS), 2016, pp. 153-154.
    [BibTeX] [Abstract] [EPrints]

    Stochastic search algorithms are general black-box optimizers. Due to their ease of use and their generality, they have recently also gained a lot of attention in operations research, machine learning and policy search. Yet, these algorithms require a lot of evaluations of the objective, scale poorly with the problem dimension, are affected by highly noisy objective functions and may converge prematurely. To alleviate these problems, we introduce a new surrogate-based stochastic search approach. We learn simple, quadratic surrogate models of the objective function. As the quality of such a quadratic approximation is limited, we do not greedily exploit the learned models. The algorithm can be misled by an inaccurate optimum introduced by the surrogate. Instead, we use information theoretic constraints to bound the ?distance? between the new and old data distribution while maximizing the objective function. Additionally the new method is able to sustain the exploration of the search distribution to avoid premature convergence. We compare our method with state of art black-box optimization methods on standard uni-modal and multi-modal optimization functions, on simulated planar robot tasks and a complex robot ball throwing task. The proposed method considerably outperforms the existing approaches.

    @inproceedings{lirolem25741,
           booktitle = {Advances in Neural Information Processing Systems (NIPS)},
               title = {Model-based relative entropy stochastic search},
              author = {A. Abdolmaleki and R. Lioutikov and N. Lua and L. Paulo Reis and J. Peters and G. Neumann},
                year = {2016},
               pages = {153--154},
             journal = {GECCO 2016 Companion - Proceedings of the 2016 Genetic and Evolutionary Computation Conference},
            keywords = {ARRAY(0x7f7859409e08)},
                 url = {http://eprints.lincoln.ac.uk/25741/},
            abstract = {Stochastic search algorithms are general black-box optimizers. Due to their ease
    of use and their generality, they have recently also gained a lot of attention in operations
    research, machine learning and policy search. Yet, these algorithms require
    a lot of evaluations of the objective, scale poorly with the problem dimension, are
    affected by highly noisy objective functions and may converge prematurely. To
    alleviate these problems, we introduce a new surrogate-based stochastic search
    approach. We learn simple, quadratic surrogate models of the objective function.
    As the quality of such a quadratic approximation is limited, we do not greedily exploit
    the learned models. The algorithm can be misled by an inaccurate optimum
    introduced by the surrogate. Instead, we use information theoretic constraints to
    bound the ?distance? between the new and old data distribution while maximizing
    the objective function. Additionally the new method is able to sustain the exploration
    of the search distribution to avoid premature convergence. We compare our
    method with state of art black-box optimization methods on standard uni-modal
    and multi-modal optimization functions, on simulated planar robot tasks and a
    complex robot ball throwing task. The proposed method considerably outperforms
    the existing approaches.}
    }
  • A. Abdolmaleki, N. Lau, L. P. Reis, and G. Neumann, “Non-parametric contextual stochastic search,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, 2016, pp. 2643-2648.
    [BibTeX] [Abstract] [EPrints]

    Stochastic search algorithms are black-box optimizer of an objective function. They have recently gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task or objective function changes slightly to adapt the solution to the new situation or the new context. In this paper, we consider the contextual stochastic search setup. Here, we want to find multiple good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective function might change slightly for each parameter vector evaluation of a task or context. Contextual algorithms have been investigated in the field of policy search, however, the search distribution typically uses a parametric model that is linear in the some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. In this paper, we propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously. In difference to existing methods, our method can also learn a context dependent covariance matrix that guides the exploration of the search process. We illustrate its performance on several non-linear contextual tasks.

    @inproceedings{lirolem25738,
              volume = {2016-N},
               month = {October},
              author = {A. Abdolmaleki and N. Lau and L.P. Reis and G. Neumann},
           booktitle = {Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on},
               title = {Non-parametric contextual stochastic search},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
               pages = {2643--2648},
                year = {2016},
            keywords = {ARRAY(0x7f78593deb00)},
                 url = {http://eprints.lincoln.ac.uk/25738/},
            abstract = {Stochastic search algorithms are black-box optimizer of an objective function. They have recently gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task or objective function changes slightly to adapt the solution to the new situation or the new context. In this paper, we consider the contextual stochastic search setup. Here, we want to find multiple good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective function might change slightly for each parameter vector evaluation of a task or context. Contextual algorithms have been investigated in the field of policy search, however, the search distribution typically uses a parametric model that is linear in the some hand-defined context features. Finding good context features is a challenging task, and hence, non-parametric methods are often preferred over their parametric counter-parts. In this paper, we propose a non-parametric contextual stochastic search algorithm that can learn a non-parametric search distribution for multiple tasks simultaneously. In difference to existing methods, our method can also learn a context dependent covariance matrix that guides the exploration of the search process. We illustrate its performance on several non-linear contextual tasks.}
    }
  • A. Abdolmaleki, N. Lau, P. L. Reis, and G. Neumann, “Contextual stochastic search,” in Genetic and Evolutionary Computation Conference GECCO 2016, 2016, pp. 29-30.
    [BibTeX] [Abstract] [EPrints]

    Stochastic search algorithms have recently also gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task changes slightly to adapt the solution to the new situation or the new context. Therefore we consider the contextual stochastic search setup. Here, we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective might change slightly for each parameter vector evaluation. In this research, we investigate the contextual stochastic search algorithms that can learn from multiple tasks simultaneously.

    @inproceedings{lirolem25679,
           booktitle = {Genetic and Evolutionary Computation Conference GECCO 2016},
               month = {July},
               title = {Contextual stochastic search},
              author = {A. Abdolmaleki and N. Lau and L. Paulo Reis and G. Neumann},
           publisher = {ACM},
                year = {2016},
               pages = {29--30},
            keywords = {ARRAY(0x7f78593e47d8)},
                 url = {http://eprints.lincoln.ac.uk/25679/},
            abstract = {Stochastic search algorithms have recently also gained a lot of attention in operations research, machine learning and policy search of robot motor skills due to their ease of use and their generality. Yet, many stochastic search algorithms require relearning if the task changes slightly to adapt the solution to the new situation or the new context. Therefore we consider the contextual stochastic search setup. Here, we want to find good parameter vectors for multiple related tasks, where each task is described by a continuous context vector. Hence, the objective might change slightly for each parameter vector evaluation. In this research, we investigate the contextual stochastic search algorithms that can learn from multiple tasks simultaneously.}
    }
  • A. Abdolmaleki, N. Lau, L. P. Reis, J. Peters, and G. Neumann, “Contextual policy search for linear and nonlinear generalization of a humanoid walking controller,” Journal of Intelligent and Robotic Systems: Theory and Applications, vol. 83, iss. 3, pp. 393-408, 2016.
    [BibTeX] [Abstract] [EPrints]

    We investigate learning of flexible robot locomotion controllers, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method which generalizes the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear policy over the contexts which call it RBF-REPS as it uses Radial Basis Functions. In order to validate our method, we perform three simulation experiments including a walking experiment using a simulated NAO humanoid robot. The robot learns a policy to choose the controller parameters for a continuous set of forward walking speeds.

    @article{lirolem25745,
              volume = {83},
              number = {3},
               month = {September},
              author = {Abbas Abdolmaleki and Nuno Lau and Luis Paulo Reis and Jan Peters and Gerhard Neumann},
               title = {Contextual policy search for linear and nonlinear generalization of a humanoid walking controller},
           publisher = {Springer},
                year = {2016},
             journal = {Journal of Intelligent and Robotic Systems: Theory and Applications},
               pages = {393--408},
            keywords = {ARRAY(0x7f78592d9a78)},
                 url = {http://eprints.lincoln.ac.uk/25745/},
            abstract = {We investigate learning of flexible robot locomotion controllers, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method which generalizes the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear policy over the contexts which call it RBF-REPS as it uses Radial Basis Functions. In order to validate our method, we perform three simulation experiments including a walking experiment using a simulated NAO humanoid robot. The robot learns a policy to choose the controller parameters for a continuous set of forward walking speeds.}
    }
  • R. Akrour, A. Abdolmaleki, H. Abdulsamad, and G. Neumann, “Model-free trajectory optimization for reinforcement learning,” in Proceedings of the International Conference on Machine Learning (ICML), 2016, pp. 4342-4352.
    [BibTeX] [Abstract] [EPrints]

    Many of the recent Trajectory Optimization algorithms alternate between local approximation of the dynamics and conservative policy update. However, linearly approximating the dynamics in order to derive the new policy can bias the update and prevent convergence to the optimal policy. In this article, we propose a new model-free algorithm that backpropagates a local quadratic time-dependent Q-Function, allowing the derivation of the policy update in closed form. Our policy update ensures exact KL-constraint satisfaction without simplifying assumptions on the system dynamics demonstrating improved performance in comparison to related Trajectory Optimization algorithms linearizing the dynamics.

    @inproceedings{lirolem25747,
              volume = {6},
               month = {June},
              author = {R. Akrour and A. Abdolmaleki and H. Abdulsamad and G. Neumann},
           booktitle = {Proceedings of the International Conference on Machine Learning (ICML)},
               title = {Model-free trajectory optimization for reinforcement learning},
             journal = {33rd International Conference on Machine Learning, ICML 2016},
               pages = {4342--4352},
                year = {2016},
            keywords = {ARRAY(0x7f78593dee90)},
                 url = {http://eprints.lincoln.ac.uk/25747/},
            abstract = {Many of the recent Trajectory Optimization algorithms
    alternate between local approximation
    of the dynamics and conservative policy update.
    However, linearly approximating the dynamics
    in order to derive the new policy can bias the update
    and prevent convergence to the optimal policy.
    In this article, we propose a new model-free
    algorithm that backpropagates a local quadratic
    time-dependent Q-Function, allowing the derivation
    of the policy update in closed form. Our policy
    update ensures exact KL-constraint satisfaction
    without simplifying assumptions on the system
    dynamics demonstrating improved performance
    in comparison to related Trajectory Optimization
    algorithms linearizing the dynamics.}
    }
  • P. Ardin, F. Peng, M. Mangan, K. Lagogiannis, and B. Webb, “Using an insect mushroom body circuit to encode route memory in complex natural environments,” PLoS Computational Biology, vol. 12, iss. 2, p. e1004683, 2016.
    [BibTeX] [Abstract] [EPrints]

    Ants, like many other animals, use visual memory to follow extended routes through complex environments, but it is unknown how their small brains implement this capability. The mushroom body neuropils have been identified as a crucial memory circuit in the insect brain, but their function has mostly been explored for simple olfactory association tasks. We show that a spiking neural model of this circuit originally developed to describe fruitfly (Drosophila melanogaster) olfactory association, can also account for the ability of desert ants (Cataglyphis velox) to rapidly learn visual routes through complex natural environments. We further demonstrate that abstracting the key computational principles of this circuit, which include one-shot learning of sparse codes, enables the theoretical storage capacity of the ant mushroom body to be estimated at hundreds of independent images.

    @article{lirolem23571,
              volume = {12},
              number = {2},
               month = {February},
              author = {Paul Ardin and Fei Peng and Michael Mangan and Konstantinos Lagogiannis and Barbara Webb},
               title = {Using an insect mushroom body circuit to encode route memory in complex natural environments},
           publisher = {Public Library of Science for International Society for Computational Biology (ISCB)},
                year = {2016},
             journal = {PLoS Computational Biology},
               pages = {e1004683},
            keywords = {ARRAY(0x7f78592e1fa0)},
                 url = {http://eprints.lincoln.ac.uk/23571/},
            abstract = {Ants, like many other animals, use visual memory to follow extended routes through complex
    environments, but it is unknown how their small brains implement this capability. The
    mushroom body neuropils have been identified as a crucial memory circuit in the insect
    brain, but their function has mostly been explored for simple olfactory association tasks. We
    show that a spiking neural model of this circuit originally developed to describe fruitfly (Drosophila
    melanogaster) olfactory association, can also account for the ability of desert ants
    (Cataglyphis velox) to rapidly learn visual routes through complex natural environments. We
    further demonstrate that abstracting the key computational principles of this circuit, which
    include one-shot learning of sparse codes, enables the theoretical storage capacity of the
    ant mushroom body to be estimated at hundreds of independent images.}
    }
  • P. B. Ardin, M. Mangan, and B. Webb, “Ant homing ability Is not diminished when traveling backwards,” Frontiers in Behavioral Neuroscience, vol. 10, 2016.
    [BibTeX] [Abstract] [EPrints]

    Ants are known to be capable of homing to their nest after displacement to a novel location. This is widely assumed to involve some form of retinotopic matching between their current view and previously experienced views. One simple algorithm proposed to explain this behavior is continuous retinotopic alignment, in which the ant constantly adjusts its heading by rotating to minimize the pixel-wise difference of its current view from all views stored while facing the nest. However, ants with large prey items will often drag them home while facing backwards. We tested whether displaced ants (Myrmecia croslandi) dragging prey could still home despite experiencing an inverted view of their surroundings under these conditions. Ants moving backwards with food took similarly direct paths to the nest as ants moving forward without food, demonstrating that continuous retinotopic alignment is not a critical component of homing. It is possible that ants use initial or intermittent retinotopic alignment, coupled with some other direction stabilizing cue that they can utilize when moving backward. However, though most ants dragging prey would occasionally look toward the nest, we observed that their heading direction was not noticeably improved afterwards. We assume ants must use comparison of current and stored images for corrections of their path, but suggest they are either able to chose the appropriate visual memory for comparison using an additional mechanism; or can make such comparisons without retinotopic alignment.

    @article{lirolem23591,
              volume = {10},
               month = {April},
               title = {Ant homing ability Is not diminished when traveling backwards},
              author = {Paul B. Ardin and Michael Mangan and Barbara Webb},
           publisher = {Frontiers Media SA},
                year = {2016},
             journal = {Frontiers in Behavioral Neuroscience},
            keywords = {ARRAY(0x7f7859409e50)},
                 url = {http://eprints.lincoln.ac.uk/23591/},
            abstract = {Ants are known to be capable of homing to their nest after displacement to a novel location. This is widely assumed to involve some form of retinotopic matching between their current view and previously experienced views. One simple algorithm proposed to explain this behavior is continuous retinotopic alignment, in which the ant constantly adjusts its heading by rotating to minimize the pixel-wise difference of its current view from all views stored while facing the nest. However, ants with large prey items will often drag them home while facing backwards. We tested whether displaced ants (Myrmecia croslandi) dragging prey could still home despite experiencing an inverted view of their surroundings under these conditions. Ants moving backwards with food took similarly direct paths to the nest as ants moving forward without food, demonstrating that continuous retinotopic alignment is not a critical component of homing. It is possible that ants use initial or intermittent retinotopic alignment, coupled with some other direction stabilizing cue that they can utilize when moving backward. However, though most ants dragging prey would occasionally look toward the nest, we observed that their heading direction was not noticeably improved afterwards. We assume ants must use comparison of current and stored images for corrections of their path, but suggest they are either able to chose the appropriate visual memory for comparison using an additional mechanism; or can make such comparisons without retinotopic alignment.}
    }
  • O. Arenz, H. Abdulsamad, and G. Neumann, “Optimal control and inverse optimal control by distribution matching,” in Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on, 2016, pp. 4046-4053.
    [BibTeX] [Abstract] [EPrints]

    Optimal control is a powerful approach to achieve optimal behavior. However, it typically requires a manual specification of a cost function which often contains several objectives, such as reaching goal positions at different time steps or energy efficiency. Manually trading-off these objectives is often difficult and requires a high engineering effort. In this paper, we present a new approach to specify optimal behavior. We directly specify the desired behavior by a distribution over future states or features of the states. For example, the experimenter could choose to reach certain mean positions with given accuracy/variance at specified time steps. Our approach also unifies optimal control and inverse optimal control in one framework. Given a desired state distribution, we estimate a cost function such that the optimal controller matches the desired distribution. If the desired distribution is estimated from expert demonstrations, our approach performs inverse optimal control. We evaluate our approach on several optimal and inverse optimal control tasks on non-linear systems using incremental linearizations similar to differential dynamic programming approaches.

    @inproceedings{lirolem25737,
              volume = {2016-N},
               month = {October},
              author = {O. Arenz and H. Abdulsamad and G. Neumann},
           booktitle = {Intelligent Robots and Systems (IROS), 2016 IEEE/RSJ International Conference on},
               title = {Optimal control and inverse optimal control by distribution matching},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
               pages = {4046--4053},
                year = {2016},
            keywords = {ARRAY(0x7f78592c3210)},
                 url = {http://eprints.lincoln.ac.uk/25737/},
            abstract = {Optimal control is a powerful approach to achieve optimal behavior. However, it typically requires a manual specification of a cost function which often contains several objectives, such as reaching goal positions at different time steps or energy efficiency. Manually trading-off these objectives is often difficult and requires a high engineering effort. In this paper, we present a new approach to specify optimal behavior. We directly specify the desired behavior by a distribution over future states or features of the states. For example, the experimenter could choose to reach certain mean positions with given accuracy/variance at specified time steps. Our approach also unifies optimal control and inverse optimal control in one framework. Given a desired state distribution, we estimate a cost function such that the optimal controller matches the desired distribution. If the desired distribution is estimated from expert demonstrations, our approach performs inverse optimal control. We evaluate our approach on several optimal and inverse optimal control tasks on non-linear systems using incremental linearizations similar to differential dynamic programming approaches.}
    }
  • F. Arvin, A. E. Turgut, T. Krajnik, and S. Yue, “Investigation of cue-based aggregation in static and dynamic environments with a mobile robot swarm,” Adaptive Behavior, vol. 24, iss. 2, pp. 102-118, 2016.
    [BibTeX] [Abstract] [EPrints]

    Aggregation is one of the most fundamental behaviors that has been studied in swarm robotic researches for more than two decades. The studies in biology revealed that environment is a preeminent factor in especially cue-based aggregation that can be defined as aggregation at a particular location which is a heat or a light source acting as a cue indicating an optimal zone. In swarm robotics, studies on cue-based aggregation mainly focused on different methods of aggregation and different parameters such as population size. Although of utmost importance, environmental effects on aggregation performance have not been studied systematically. In this paper, we study the effects of different environmental factors; size, texture and number of cues in a static setting and moving cues in a dynamic setting using real robots. We used aggregation time and size of the aggregate as the two metrics to measure aggregation performance. We performed real robot experiments with different population sizes and evaluated the performance of aggregation using the defined metrics. We also proposed a probabilistic aggregation model and predicted the aggregation performance accurately in most of the settings. The results of the experiments show that environmental conditions affect the aggregation performance considerably and have to be studied in depth.

    @article{lirolem22466,
              volume = {24},
              number = {2},
               month = {April},
              author = {Farshad Arvin and Ali Emre Turgut and Tomas Krajnik and Shigang Yue},
               title = {Investigation of cue-based aggregation in static and dynamic environments with a mobile robot swarm},
           publisher = {SAGE},
                year = {2016},
             journal = {Adaptive Behavior},
               pages = {102--118},
            keywords = {ARRAY(0x7f78593a53b0)},
                 url = {http://eprints.lincoln.ac.uk/22466/},
            abstract = {Aggregation is one of the most fundamental behaviors that has been studied in swarm robotic researches for more than two decades. The studies in biology revealed that environment is a preeminent factor in especially cue-based aggregation that can be defined as aggregation at a particular location which is a heat or a light source acting as a cue indicating an optimal zone. In swarm robotics, studies on cue-based aggregation mainly focused on different methods of aggregation and different parameters such as population size. Although of utmost importance, environmental effects on aggregation performance have not been studied systematically. In this paper, we study the effects of different environmental factors; size, texture and number of cues in a static setting and moving cues in a dynamic setting using real robots. We used aggregation time and size of the aggregate as the two metrics to measure aggregation performance. We performed real robot experiments with different population sizes and evaluated the performance of aggregation using the defined metrics. We also proposed a probabilistic aggregation model and predicted the aggregation performance accurately in most of the settings. The results of the experiments show that environmental conditions affect the aggregation performance considerably and have to be studied in depth.}
    }
  • G. Broughton, T. Krajnik, M. Fernandez-carmona, G. Cielniak, and N. Bellotto, “RFID-based Object Localisation with a Mobile Robot to Assist the Elderly with Mild Cognitive Impairments,” in International Workshop on Intelligent Environments Supporting Healthcare and Well-being (WISHWell), 2016.
    [BibTeX] [Abstract] [EPrints]

    Mild Cognitive Impairments (MCI) disrupt the quality of life and reduce the independence of many elderly at home. People with MCI can increasingly become forgetful, hence solutions to help them ?finding lost objects are useful. This paper presents a framework for mobile robots to localise objects in a domestic environment using Radio Frequency Identification (RFID) technology. In particular, it describes the development of a new library for interacting with RFID readers, readily available for the Robot Operating System (ROS), and introduces some methods for its application to RFID-based object localisation with a single antenna. The framework adopts occupancy grids to create a probabilistic representations of tags location in the environment. A robot traversing the environment can then make use of this framework to keep an internal record of where objects were last spotted, and where they are most likely to be at any given point in time. Some preliminary results are presented, together with directions for future research.

    @inproceedings{lirolem23298,
           booktitle = {International Workshop on Intelligent Environments Supporting Healthcare and Well-being (WISHWell)},
               month = {September},
               title = {RFID-based Object Localisation with a Mobile Robot to Assist the Elderly with Mild Cognitive Impairments},
              author = {George Broughton and Tomas Krajnik and Manuel Fernandez-carmona and Grzegorz Cielniak and Nicola Bellotto},
                year = {2016},
            keywords = {ARRAY(0x7f78593e2b98)},
                 url = {http://eprints.lincoln.ac.uk/23298/},
            abstract = {Mild Cognitive Impairments (MCI) disrupt the quality of life and reduce the independence of many elderly at home. People with MCI can increasingly become forgetful, hence solutions to help them ?finding lost objects are useful. This paper presents a framework for mobile robots to localise objects in a domestic environment using Radio Frequency Identification (RFID) technology. In particular, it describes the development of a new library for interacting with RFID readers, readily available for the Robot Operating System (ROS), and introduces some methods for its application to RFID-based object localisation with a single antenna. The framework adopts occupancy grids to create a probabilistic representations of tags location in the environment. A robot traversing the environment can then make use of this framework to keep an internal record of where objects were last spotted, and where they are most likely to be at any given point in time. Some preliminary results are presented, together with directions for future research.}
    }
  • W. Chen, C. Xiong, and S. Yue, “On configuration trajectory formation in spatiotemporal profile for reproducing human hand reaching movement,” IEEE Transactions on Cybernetics, vol. 46, iss. 3, 2016.
    [BibTeX] [Abstract] [EPrints]

    Most functional reaching activities in daily living generally require a hand to reach the functional position in appropriate orientation with invariant spatiotemporal profile. Effectively reproducing such spatiotemporal feature of hand configuration trajectory in real time is essential to understand the human motor control and plan human-like motion on anthropomorphic robotic arm. However, there are no novel computational models in literature toward reproducing hand configuration-to-configuration movement in spatiotemporal profile. In response to the problem, this paper presents a computational framework for hand configuration trajectory formation based on hierarchical principle of human motor control. The composite potential field is constructed on special Euclidean Group to induce time-varying configuration toward target. The dynamic behavior of hand is described by a second-order kinematic model to produce the external representation of high-level motor control. The multivariate regression relation between intrinsic and extrinsic coordinates of arm, is statistically analyzed for determining the arm orientation in real time, which produces the external representation of low-level motor control. The proposed method is demonstrated in an anthropomorphic arm by performing several highly curved self-reaching movements. The generated configuration trajectories are compared with actual human movement in spatiotemporal profile to validate the proposed method.

    @article{lirolem17880,
              volume = {46},
              number = {3},
               month = {March},
              author = {Wenbin Chen and Caihua Xiong and Shigang Yue},
               title = {On configuration trajectory formation in spatiotemporal profile for reproducing human hand reaching movement},
           publisher = {IEEE},
             journal = {IEEE Transactions on Cybernetics},
                year = {2016},
            keywords = {ARRAY(0x7f78591305b8)},
                 url = {http://eprints.lincoln.ac.uk/17880/},
            abstract = {Most functional reaching activities in daily living generally require a hand to reach the functional position in appropriate orientation with invariant spatiotemporal profile. Effectively reproducing such spatiotemporal feature of hand configuration trajectory in real time is essential to understand the human motor control and plan human-like motion on anthropomorphic robotic arm. However, there are no novel computational models in literature toward reproducing hand configuration-to-configuration movement in spatiotemporal profile. In response to the problem, this paper presents a computational framework for hand configuration trajectory formation based on hierarchical principle of human motor control. The composite potential field is constructed on special Euclidean Group to induce time-varying configuration toward target. The dynamic behavior of hand is described by a second-order kinematic model to produce the external representation of high-level motor control. The multivariate regression relation between intrinsic and extrinsic coordinates of arm, is statistically analyzed for determining the arm orientation in real time, which produces the external representation of low-level motor control. The proposed method is demonstrated in an anthropomorphic arm by performing several highly curved self-reaching movements. The generated configuration trajectories are compared with actual human movement in spatiotemporal profile to validate the proposed method.}
    }
  • A. Coninx, P. Baxter, E. Oleari, S. Bellini, B. Bierman, O. B. Henkemans, L. Canamero, P. Cosi, V. Enescu, R. R. Espinoza, A. Hiolle, R. Humbert, B. Kiefer, I. Kruijff-korbayova, R. Looije, M. Mosconi, M. Neerincx, G. Paci, G. Patsis, C. Pozzi, F. Sacchitelli, H. Sahli, A. Sanna, G. Sommavilla, F. Tesser, Y. Demiris, and T. Belpaeme, “Towards long-term social child-robot interaction: using multi-activity switching to engage young users,” Journal of Human-Robot Interaction, vol. 5, iss. 1, pp. 32-67, 2016.
    [BibTeX] [Abstract] [EPrints]

    Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI.

    @article{lirolem23074,
              volume = {5},
              number = {1},
               title = {Towards long-term social child-robot interaction: using multi-activity switching to engage young users},
              author = {Alexandre Coninx and Paul Baxter and Elettra Oleari and Sara Bellini and Bert Bierman and Olivier Blanson Henkemans and Lola Canamero and Piero Cosi and Valentin Enescu and Raquel Ros Espinoza and Antoine Hiolle and Remi Humbert and Bernd Kiefer and Ivana Kruijff-korbayova and Rosemarijn Looije and Marco Mosconi and Mark Neerincx and Giulio Paci and Georgios Patsis and Clara Pozzi and Francesca Sacchitelli and Hichem Sahli and Alberto Sanna and Giacomo Sommavilla and Fabio Tesser and Yiannis Demiris and Tony Belpaeme},
                year = {2016},
               pages = {32--67},
             journal = {Journal of Human-Robot Interaction},
            keywords = {ARRAY(0x7f78592e22e8)},
                 url = {http://eprints.lincoln.ac.uk/23074/},
            abstract = {Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI.}
    }
  • C. Coppola, T. Krajnik, T. Duckett, and N. Bellotto, “Learning temporal context for activity recognition,” in European Conference on Artificial Intelligence (ECAI), 2016.
    [BibTeX] [Abstract] [EPrints]

    We investigate how incremental learning of long-term human activity patterns improves the accuracy of activity classification over time. Rather than trying to improve the classification methods themselves, we assume that they can take into account prior probabilities of activities occurring at a particular time. We use the classification results to build temporal models that can provide these priors to the classifiers. As our system gradually learns about typical patterns of human activities, the accuracy of activity classification improves, which results in even more accurate priors. Two datasets collected over several months containing hand-annotated activity in residential and office environments were chosen to evaluate the approach. Several types of temporal models were evaluated for each of these datasets. The results indicate that incremental learning of daily routines leads to a significant improvement in activity classification.

    @inproceedings{lirolem23297,
           booktitle = {European Conference  on Artificial Intelligence (ECAI)},
               month = {August},
               title = {Learning temporal context for activity recognition},
              author = {Claudio Coppola and Tomas Krajnik and Tom Duckett and Nicola Bellotto},
                year = {2016},
            keywords = {ARRAY(0x7f78592c62f0)},
                 url = {http://eprints.lincoln.ac.uk/23297/},
            abstract = {We investigate how incremental learning of long-term human activity patterns improves the accuracy of activity classification over time. Rather than trying to improve the classification methods themselves, we assume that they can take into account prior probabilities of activities occurring at a particular time. We use the classification results to build temporal models that can provide these priors to the classifiers. As our system gradually learns about typical patterns of human activities, the accuracy of activity classification improves, which results in even more accurate priors. Two datasets collected over several months containing hand-annotated activity in residential and office environments were chosen to evaluate the approach. Several types of temporal models were evaluated for each of these datasets. The results indicate that incremental learning of daily routines leads to a significant improvement in activity classification.}
    }
  • C. Coppola, D. Faria, U. Nunes, and N. Bellotto, “Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.
    [BibTeX] [Abstract] [EPrints]

    Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of classifiers; and (3) provide a public dataset with RGB-D data of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using a modified dynamic Bayesian mixture model designed to merge features with different semantics and also with proximity priors, the proposed framework can correctly recognize social activities in different situations, e.g. using data from one or two individuals.

    @inproceedings{lirolem23425,
           booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
               month = {October},
               title = {Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data},
              author = {Claudio Coppola and Diego Faria and Urbano Nunes and Nicola Bellotto},
           publisher = {IEEE},
                year = {2016},
            keywords = {ARRAY(0x7f78592c5f60)},
                 url = {http://eprints.lincoln.ac.uk/23425/},
            abstract = {Social activity based on body motion is a key feature for non-verbal and physical behavior defined as function for communicative signal and social interaction between individuals. Social activity recognition is important to study human-human communication and also human-robot interaction. Based on that, this research has threefold goals: (1) recognition of social behavior (e.g. human-human interaction) using a probabilistic approach that merges spatio-temporal features from individual bodies and social features from the relationship between two individuals; (2) learn priors based on physical proximity between individuals during an interaction using proxemics theory to feed a probabilistic ensemble of classifiers; and (3) provide a public dataset with RGB-D data
    of social daily activities including risk situations useful to test approaches for assisted living, since this type of dataset is still missing. Results show that using a modified dynamic Bayesian mixture model designed to merge features with different semantics and also with proximity priors, the proposed framework can correctly recognize social activities in different situations, e.g. using data from one or two individuals.}
    }
  • H. Cuayahuitl, S. Yu, A. Williamson, and J. Carse, “Deep reinforcement learning for multi-domain dialogue systems,” in NIPS Workshop on Deep Reinforcement Learning, 2016.
    [BibTeX] [Abstract] [EPrints]

    Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems. We propose a method for multi-domain dialogue policy learning—termed NDQN, and apply it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that our proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems.

    @inproceedings{lirolem25935,
              volume = {abs/16},
               month = {December},
              author = {Heriberto Cuayahuitl and Seunghak Yu and Ashley Williamson and Jacob Carse},
           booktitle = {NIPS Workshop on Deep Reinforcement Learning},
               title = {Deep reinforcement learning for multi-domain dialogue systems},
           publisher = {arXiv},
             journal = {CoRR},
                year = {2016},
            keywords = {ARRAY(0x7f78592c6248)},
                 url = {http://eprints.lincoln.ac.uk/25935/},
            abstract = {Standard deep reinforcement learning methods such as Deep Q-Networks (DQN) for multiple tasks (domains) face scalability problems. We propose a method for multi-domain dialogue policy learning---termed NDQN, and apply it to an information-seeking spoken dialogue system in the domains of restaurants and hotels. Experimental results comparing DQN (baseline) versus NDQN (proposed) using simulations report that our proposed method exhibits better scalability and is promising for optimising the behaviour of multi-domain dialogue systems.}
    }
  • H. Cuayahuitl, G. Couly, and C. Olalainty, “Training an interactive humanoid robot using multimodal deep reinforcement learning,” in NIPS Workshop on Deep Reinforcement Learning, 2016.
    [BibTeX] [Abstract] [EPrints]

    Training robots to perceive, act and communicate using multiple modalities still represents a challenging problem, particularly if robots are expected to learn efficiently from small sets of example interactions. We describe a learning approach as a step in this direction, where we teach a humanoid robot how to play the game of noughts and crosses. Given that multiple multimodal skills can be trained to play this game, we focus our attention to training the robot to perceive the game, and to interact in this game. Our multimodal deep reinforcement learning agent perceives multimodal features and exhibits verbal and non-verbal actions while playing. Experimental results using simulations show that the robot can learn to win or draw up to 98\% of the games. A pilot test of the proposed multimodal system for the targeted game—integrating speech, vision and gestures—reports that reasonable and fluent interactions can be achieved using the proposed approach.

    @inproceedings{lirolem25937,
              volume = {abs/16},
               month = {December},
              author = {Heriberto Cuayahuitl and Guillaume Couly and Clement Olalainty},
           booktitle = {NIPS Workshop on Deep Reinforcement Learning},
               title = {Training an interactive humanoid robot using multimodal deep reinforcement learning},
           publisher = {arXiv},
             journal = {CoRR},
                year = {2016},
            keywords = {ARRAY(0x7f78592ddb30)},
                 url = {http://eprints.lincoln.ac.uk/25937/},
            abstract = {Training robots to perceive, act and communicate using multiple modalities still represents a challenging problem, particularly if robots are expected to learn efficiently from small sets of example interactions. We describe a learning approach as a step in this direction, where we teach a humanoid robot how to play the game of noughts and crosses. Given that multiple multimodal skills can be trained to play this game, we focus our attention to training the robot to perceive the game, and to interact in this game. Our multimodal deep reinforcement learning agent perceives multimodal features and exhibits verbal and non-verbal actions while playing. Experimental results using simulations show that the robot can learn to win or draw up to 98\% of the games. A pilot test of the proposed multimodal system for the targeted game---integrating speech, vision and gestures---reports that reasonable and fluent interactions can be achieved using the proposed approach.}
    }
  • C. Daniel, G. Neumann, O. Kroemer, and J. Peters, “Hierarchical relative entropy policy search,” Journal of Machine Learning Research, vol. 17, pp. 1-50, 2016.
    [BibTeX] [Abstract] [EPrints]

    Many reinforcement learning (RL) tasks, especially in robotics, consist of multiple sub-tasks that are strongly structured. Such task structures can be exploited by incorporating hierarchical policies that consist of gating networks and sub-policies. However, this concept has only been partially explored for real world settings and complete methods, derived from first principles, are needed. Real world settings are challenging due to large and continuous state-action spaces that are prohibitive for exhaustive sampling methods. We define the problem of learning sub-policies in continuous state action spaces as finding a hierarchical policy that is composed of a high-level gating policy to select the low-level sub-policies for execution by the agent. In order to efficiently share experience with all sub-policies, also called inter-policy learning, we treat these sub-policies as latent variables which allows for distribution of the update information between the sub-policies. We present three different variants of our algorithm, designed to be suitable for a wide variety of real world robot learning tasks and evaluate our algorithms in two real robot learning scenarios as well as several simulations and comparisons.

    @article{lirolem25743,
              volume = {17},
               month = {June},
              author = {C. Daniel and G. Neumann and O. Kroemer and J. Peters},
               title = {Hierarchical relative entropy policy search},
           publisher = {Massachusetts Institute of Technology Press (MIT Press) / Microtome Publishing},
             journal = {Journal of Machine Learning Research},
               pages = {1--50},
                year = {2016},
            keywords = {ARRAY(0x7f78592c60e0)},
                 url = {http://eprints.lincoln.ac.uk/25743/},
            abstract = {Many reinforcement learning (RL) tasks, especially in robotics, consist of multiple sub-tasks that
    are strongly structured. Such task structures can be exploited by incorporating hierarchical policies
    that consist of gating networks and sub-policies. However, this concept has only been partially explored
    for real world settings and complete methods, derived from first principles, are needed. Real
    world settings are challenging due to large and continuous state-action spaces that are prohibitive
    for exhaustive sampling methods. We define the problem of learning sub-policies in continuous
    state action spaces as finding a hierarchical policy that is composed of a high-level gating policy to
    select the low-level sub-policies for execution by the agent. In order to efficiently share experience
    with all sub-policies, also called inter-policy learning, we treat these sub-policies as latent variables
    which allows for distribution of the update information between the sub-policies. We present three
    different variants of our algorithm, designed to be suitable for a wide variety of real world robot
    learning tasks and evaluate our algorithms in two real robot learning scenarios as well as several
    simulations and comparisons.}
    }
  • C. Daniel, H. van Hoof, J. Peters, and G. Neumann, “Probabilistic inference for determining options in reinforcement learning,” Machine Learning, vol. 104, iss. 2-3, pp. 337-357, 2016.
    [BibTeX] [Abstract] [EPrints]

    Tasks that require many sequential decisions or complex solutions are hard to solve using conventional reinforcement learning algorithms. Based on the semi Markov decision process setting (SMDP) and the option framework, we propose a model which aims to alleviate these concerns. Instead of learning a single monolithic policy, the agent learns a set of simpler sub-policies as well as the initiation and termination probabilities for each of those sub-policies. While existing option learning algorithms frequently require manual specification of components such as the sub-policies, we present an algorithm which infers all relevant components of the option framework from data. Furthermore, the proposed approach is based on parametric option representations and works well in combination with current policy search methods, which are particularly well suited for continuous real-world tasks. We present results on SMDPs with discrete as well as continuous state-action spaces. The results show that the presented algorithm can combine simple sub-policies to solve complex tasks and can improve learning performance on simpler tasks.

    @article{lirolem25739,
              volume = {104},
              number = {2-3},
               month = {September},
              author = {C. Daniel and H. van Hoof and J. Peters and G. Neumann},
               title = {Probabilistic inference for determining options in reinforcement learning},
           publisher = {Springer},
                year = {2016},
             journal = {Machine Learning},
               pages = {337--357},
            keywords = {ARRAY(0x7f78592c6428)},
                 url = {http://eprints.lincoln.ac.uk/25739/},
            abstract = {Tasks that require many sequential decisions or complex solutions are hard to solve using conventional reinforcement learning algorithms. Based on the semi Markov decision process setting (SMDP) and the option framework, we propose a model which aims to alleviate these concerns. Instead of learning a single monolithic policy, the agent learns a set of simpler sub-policies as well as the initiation and termination probabilities for each of those sub-policies. While existing option learning algorithms frequently require manual specification of components such as the sub-policies, we present an algorithm which infers all relevant components of the option framework from data. Furthermore, the proposed approach is based on parametric option representations and works well in combination with current policy search methods, which are particularly well suited for continuous real-world tasks. We present results on SMDPs with discrete as well as continuous state-action spaces. The results show that the presented algorithm can combine simple sub-policies to solve complex tasks and can improve learning performance on simpler tasks.}
    }
  • N. Dethlefs, H. Hastie, H. Cuayahuitl, Y. Yu, V. Rieser, and O. Lemon, “Information density and overlap in spoken dialogue,” Computer Speech & Language, vol. 37, pp. 82-97, 2016.
    [BibTeX] [Abstract] [EPrints]

    Incremental dialogue systems are often perceived as more responsive and natural because they are able to address phenomena of turn-taking and overlapping speech, such as backchannels or barge-ins. Previous work in this area has often identified distinctive prosodic features, or features relating to syntactic or semantic completeness, as marking appropriate places of turn-taking. In a separate strand of work, psycholinguistic studies have established a connection between information density and prominence in language–the less expected a linguistic unit is in a particular context, the more likely it is to be linguistically marked. This has been observed across linguistic levels, including the prosodic, which plays an important role in predicting overlapping speech. In this article, we explore the hypothesis that information density (ID) also plays a role in turn-taking. Specifically, we aim to show that humans are sensitive to the peaks and troughs of information density in speech, and that overlapping speech at ID troughs is perceived as more acceptable than overlaps at ID peaks. To test our hypothesis, we collect human ratings for three models of generating overlapping speech based on features of: (1) prosody and semantic or syntactic completeness, (2) information density, and (3) both types of information. Results show that over 50\% of users preferred the version using both types of features, followed by a preference for information density features alone. This indicates a clear human sensitivity to the effects of information density in spoken language and provides a strong motivation to adopt this metric for the design, development and evaluation of turn-taking modules in spoken and incremental dialogue systems.

    @article{lirolem22216,
              volume = {37},
               month = {May},
              author = {Nina Dethlefs and Helen Hastie and Heriberto Cuayahuitl and Yanchao Yu and Verena Rieser and Oliver Lemon},
               title = {Information density and overlap in spoken dialogue},
           publisher = {Elsevier for International Speech Communication Association (ISCA)},
             journal = {Computer Speech \& Language},
               pages = {82--97},
                year = {2016},
            keywords = {ARRAY(0x7f78592d48f8)},
                 url = {http://eprints.lincoln.ac.uk/22216/},
            abstract = {Incremental dialogue systems are often perceived as more responsive and natural because they are able to address phenomena of turn-taking and overlapping speech, such as backchannels or barge-ins. Previous work in this area has often identified distinctive prosodic features, or features relating to syntactic or semantic completeness, as marking appropriate places of turn-taking. In a separate strand of work, psycholinguistic studies have established a connection between information density and prominence in language{--}the less expected a linguistic unit is in a particular context, the more likely it is to be linguistically marked. This has been observed across linguistic levels, including the prosodic, which plays an important role in predicting overlapping speech.
    
    In this article, we explore the hypothesis that information density (ID) also plays a role in turn-taking. Specifically, we aim to show that humans are sensitive to the peaks and troughs of information density in speech, and that overlapping speech at ID troughs is perceived as more acceptable than overlaps at ID peaks. To test our hypothesis, we collect human ratings for three models of generating overlapping speech based on features of: (1) prosody and semantic or syntactic completeness, (2) information density, and (3) both types of information. Results show that over 50\% of users preferred the version using both types of features, followed by a preference for information density features alone. This indicates a clear human sensitivity to the effects of information density in spoken language and provides a strong motivation to adopt this metric for the design, development and evaluation of turn-taking modules in spoken and incremental dialogue systems.}
    }
  • P. Dickinson, O. Szymanezyk, G. Cielniak, and M. Mannion, “Indoor positioning of shoppers using a network of bluetooth low energy beacons,” in 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 4-7 October 2016, Alcalá de Henares, Spain, 2016.
    [BibTeX] [Abstract] [EPrints]

    In this paper we present our work on the indoor positioning of users (shoppers), using a network of Bluetooth Low Energy (BLE) beacons deployed in a large wholesale shopping store. Our objective is to accurately determine which product sections a user is adjacent to while traversing the store, using RSSI readings from multiple beacons, measured asynchronously on a standard commercial mobile device. We further wish to leverage the store layout (which imposes natural constraints on the movement of users) and the physical configuration of the beacon network, to produce a robust and efficient solution. We start by describing our application context and hardware configuration, and proceed to introduce our node-graph model of user location. We then describe our experimental work which begins with an investigation of signal characteristics along and across aisles. We propose three methods of localization, using a ?nearest-beacon? approach as a base-line; exponentially averaged weighted range estimates; and a particle-filter method based on the RSSI attenuation model and Gaussian-noise. Our results demonstrate that the particle filter method significantly out-performs the others. Scalability also makes this method ideal for applications run on mobile devices with more limited computational capabilities

    @inproceedings{lirolem24589,
           booktitle = {2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN), 4-7 October 2016, Alcal{\'a} de Henares, Spain},
               month = {October},
               title = {Indoor positioning of shoppers using a network of bluetooth low energy beacons},
              author = {Patrick Dickinson and Olivier Szymanezyk and Grzegorz Cielniak and Mike Mannion},
           publisher = {IEEE Xplore},
                year = {2016},
            keywords = {ARRAY(0x7f78593d7df8)},
                 url = {http://eprints.lincoln.ac.uk/24589/},
            abstract = {In this paper we present our work on the indoor positioning of users (shoppers), using a network of Bluetooth Low Energy (BLE) beacons deployed in a large wholesale shopping store. Our objective is to accurately determine which product sections a user is adjacent to while traversing the store, using RSSI readings from multiple beacons, measured asynchronously on a standard commercial mobile device. We further wish to leverage the store layout (which imposes natural constraints on the movement of users) and the physical configuration of the beacon network, to produce a robust and efficient solution. We start by describing our application context and hardware configuration, and proceed to introduce our node-graph model of user location. We then describe our experimental work which begins with an investigation of signal characteristics along and across aisles. We propose three methods of localization, using a ?nearest-beacon? approach as a base-line; exponentially averaged weighted range estimates; and a particle-filter method based on the RSSI attenuation model and Gaussian-noise. Our results demonstrate that the particle filter method significantly out-performs the others. Scalability also makes this method ideal for applications run on mobile devices with more limited computational capabilities}
    }
  • C. Dondrup and M. Hanheide, “Qualitative constraints for human-aware robot navigation using Velocity Costmaps,” in 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), 2016, pp. 586-592.
    [BibTeX] [Abstract] [EPrints]

    In this work, we propose the combination of a state-of-the-art sampling-based local planner with so-called Velocity Costmaps to achieve human-aware robot navigation. Instead of introducing humans as ?special obstacles? into the representation of the environment, we restrict the sample space of a ?Dynamic Window Approach? local planner to only allow trajectories based on a qualitative description of the future unfolding of the encounter. To achieve this, we use a Bayesian temporal model based on a Qualitative Trajectory Calculus to represent the mutual navigation intent of human and robot, and translate these descriptors into sample space constraints for trajectory generation. We show how to learn these models from demonstration and evaluate our approach against standard Gaussian cost models in simulation and in real-world using a non-holonomic mobile robot. Our experiments show that our approach exceeds the performance and safety of the Gaussian models in pass-by and path crossing situations.

    @inproceedings{lirolem27957,
           booktitle = {2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN)},
               month = {August},
               title = {Qualitative constraints for human-aware robot navigation using Velocity Costmaps},
              author = {Christian Dondrup and Marc Hanheide},
           publisher = {IEEE},
                year = {2016},
               pages = {586--592},
            keywords = {ARRAY(0x7f78593e4b38)},
                 url = {http://eprints.lincoln.ac.uk/27957/},
            abstract = {In this work, we propose the combination of a state-of-the-art sampling-based local planner with so-called Velocity Costmaps to achieve human-aware robot navigation. Instead of introducing humans as ?special obstacles? into the representation of the environment, we restrict the sample space of a ?Dynamic Window Approach? local planner to only allow trajectories based on a qualitative description of the future unfolding of the encounter. To achieve this, we use a Bayesian temporal model based on a Qualitative Trajectory Calculus to represent the mutual navigation intent of human and robot, and translate these descriptors into sample space constraints for trajectory generation. We show how to learn these models from demonstration and evaluate our approach against standard Gaussian cost models in simulation and in real-world using a non-holonomic mobile robot. Our experiments show that our approach exceeds the performance and safety of the Gaussian models in pass-by and path crossing situations.}
    }
  • M. Ewerton, G. Maeda, G. Neumann, V. Kisner, G. Kollegger, J. Wiemeyer, and J. Peters, “Movement primitives with multiple phase parameters,” in Robotics and Automation (ICRA), 2016 IEEE International Conference on, 2016, pp. 201-206.
    [BibTeX] [Abstract] [EPrints]

    Movement primitives are concise movement representations that can be learned from human demonstrations, support generalization to novel situations and modulate the speed of execution of movements. The speed modulation mechanisms proposed so far are limited though, allowing only for uniform speed modulation or coupling changes in speed to local measurements of forces, torques or other quantities. Those approaches are not enough when dealing with general velocity constraints. We present a movement primitive formulation that can be used to non-uniformly adapt the speed of execution of a movement in order to satisfy a given constraint, while maintaining similarity in shape to the original trajectory. We present results using a 4-DoF robot arm in a minigolf setup.

    @inproceedings{lirolem25742,
              volume = {2016-J},
               month = {June},
              author = {M. Ewerton and G. Maeda and G. Neumann and V. Kisner and G. Kollegger and J. Wiemeyer and J. Peters},
           booktitle = {Robotics and Automation (ICRA), 2016 IEEE International Conference on},
               title = {Movement primitives with multiple phase parameters},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {201--206},
                year = {2016},
            keywords = {ARRAY(0x7f78592c62d8)},
                 url = {http://eprints.lincoln.ac.uk/25742/},
            abstract = {Movement primitives are concise movement representations that can be learned from human demonstrations, support generalization to novel situations and modulate the speed of execution of movements. The speed modulation mechanisms proposed so far are limited though, allowing only for uniform speed modulation or coupling changes in speed to local measurements of forces, torques or other quantities. Those approaches are not enough when dealing with general velocity constraints. We present a movement primitive formulation that can be used to non-uniformly adapt the speed of execution of a movement in order to satisfy a given constraint, while maintaining similarity in shape to the original trajectory. We present results using a 4-DoF robot arm in a minigolf setup.}
    }
  • J. P. Fentanes, T. Krajnik, M. Hanheide, and T. Duckett, “Persistent localization and life-long mapping in changing environments using the frequency map enhancement,” in IEEE/RSJ International Conference on Intelligent Robots ans Systems (IROS), 2016.
    [BibTeX] [Abstract] [EPrints]

    We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments. The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future. During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps used by the robot’s localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.

    @inproceedings{lirolem24088,
           booktitle = {IEEE/RSJ International Conference on Intelligent Robots ans Systems (IROS)},
               month = {October},
               title = {Persistent localization and life-long mapping in changing environments using the frequency map enhancement},
              author = {Jaime Pulido Fentanes and Tomas Krajnik and Marc Hanheide and Tom Duckett},
           publisher = {IEEE},
                year = {2016},
            keywords = {ARRAY(0x7f78592eacd0)},
                 url = {http://eprints.lincoln.ac.uk/24088/},
            abstract = {We present a lifelong mapping and localisation system for long-term autonomous operation of mobile robots in changing environments.
    The core of the system is a spatio-temporal occupancy grid that explicitly represents the persistence and periodicity of the individual cells and can predict the probability of their occupancy in the future.
    During navigation, our robot builds temporally local maps and integrates then into the global spatio-temporal grid. Through re-observation of the same locations, the spatio-temporal grid learns the long-term environment dynamics and gains the ability to predict the future environment states. This predictive ability allows to generate time-specific 2d maps  used by the robot's localisation and planning modules. By analysing data from a long-term deployment of the robot in a human-populated environment, we show that the proposed  representation improves localisation accuracy and the efficiency of path planning. We also show how to integrate the method into the ROS navigation stack for use by other roboticists.}
    }
  • M. Fernandez-Carmona and N. Bellotto, “On-line inference comparison with Markov Logic Network engines for activity recognition in AAL environments,” in IEEE International Conference on Intelligent Environments, 2016.
    [BibTeX] [Abstract] [EPrints]

    We address possible solutions for a practical application of Markov Logic Networks to online activity recognition, based on domotic sensors, to be used for monitoring elderly with mild cognitive impairments. Our system has to provide responsive information about user activities throughout the day, so different inference engines are tested. We use an abstraction layer to gather information from commercial domotic sensors. Sensor events are stored using a non-relational database. Using this database, evidences are built to query a logic network about current activities. Markov Logic Networks are able to deal with uncertainty while keeping a structured knowledge. This makes them a suitable tool for ambient sensors based inference. However, in their previous application, inferences are usually made offline. Time is a relevant constrain in our system and hence logic networks are designed here accordingly. We compare in this work different engines to model a Markov Logic Network suitable for such circumstances. Results show some insights about how to design a low latency logic network and which kind of solutions should be avoided.

    @inproceedings{lirolem23189,
           booktitle = {IEEE International Conference on Intelligent Environments},
               month = {September},
               title = {On-line inference comparison with Markov Logic Network engines for activity recognition in AAL environments},
              author = {Manuel Fernandez-Carmona and Nicola Bellotto},
           publisher = {IEEE},
                year = {2016},
            keywords = {ARRAY(0x7f78592c2b98)},
                 url = {http://eprints.lincoln.ac.uk/23189/},
            abstract = {We address possible solutions for a practical application of Markov Logic Networks to online activity recognition, based on domotic sensors, to be used for monitoring elderly with mild cognitive impairments. Our system has to provide responsive information about user activities throughout the day, so different inference engines are tested. We use an abstraction layer to gather information from commercial domotic sensors.  Sensor events are stored using a non-relational database. Using this database, evidences are built to query a logic network about current activities. Markov Logic Networks are able to deal with uncertainty while keeping a structured knowledge. This makes them a suitable tool for ambient sensors based inference. However, in their previous application, inferences are usually made offline. Time is a relevant constrain in our system and hence logic networks are designed here accordingly. We compare in this work different engines to model a Markov Logic Network suitable for such circumstances. Results show some insights about how to design a low latency logic network and which kind of solutions should be avoided.}
    }
  • Q. Fu, S. Yue, and C. Hu, “Bio-inspired collision detector with enhanced selectivity for ground robotic vision system,” in 27th British Machine Vision Conference, 2016.
    [BibTeX] [Abstract] [EPrints]

    There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ?rst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ?rst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing ef?cient visual sensor, and realizing the revealed speci?c characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.

    @inproceedings{lirolem24941,
           booktitle = {27th British Machine Vision Conference},
               month = {September},
               title = {Bio-inspired collision detector with enhanced selectivity for ground robotic vision system},
              author = {Qinbing Fu and Shigang Yue and Cheng Hu},
                year = {2016},
            keywords = {ARRAY(0x7f78592c33f0)},
                 url = {http://eprints.lincoln.ac.uk/24941/},
            abstract = {There are many ways of building collision-detecting systems. In this paper, we propose a novel collision selective visual neural network inspired by LGMD2 neurons in the juvenile locusts. Such collision-sensitive neuron matures early in the ?rst-aged or even hatching locusts, and is only selective to detect looming dark objects against bright background in depth, represents swooping predators, a situation which is similar to ground robots or vehicles. However, little has been done on modeling LGMD2, let alone its potential applications in robotics and other vision-based areas. Compared to other collision detectors, our major contributions are ?rst, enhancing the collision selectivity in a bio-inspired way, via constructing a computing ef?cient visual sensor, and realizing the revealed speci?c characteristic sofLGMD2. Second, we applied the neural network to help rearrange path navigation of an autonomous ground miniature robot in an arena. We also examined its neural properties through systematic experiments challenged against image streams from a visual sensor of the micro-robot.}
    }
  • Y. Gatsoulis, M. Alomari, C. Burbridge, C. Dondrup, P. Duckworth, P. Lightbody, M. Hanheide, N. Hawes, D. C. Hogg, and A. G. Cohn, “QSRlib: a software library for online acquisition of qualitative spatial relations from video,” in 29th International Workshop on Qualitative Reasoning (QR16), at IJCAI-16, 2016.
    [BibTeX] [Abstract] [EPrints]

    There is increasing interest in using Qualitative Spatial Relations as a formalism to abstract from noisy and large amounts of video data in order to form high level conceptualisations, e.g. of activities present in video. We present a library to support such work. It is compatible with the Robot Operating System (ROS) but can also be used stand alone. A number of QSRs are built in; others can be easily added.

    @inproceedings{lirolem24853,
           booktitle = {29th International Workshop on Qualitative Reasoning (QR16), at IJCAI-16},
               month = {July},
               title = {QSRlib: a software library for online acquisition of qualitative spatial relations from video},
              author = {Y. Gatsoulis and M. Alomari and C. Burbridge and C. Dondrup and P. Duckworth and P. Lightbody and M. Hanheide and N. Hawes and D. C. Hogg and A. G. Cohn},
                year = {2016},
            keywords = {ARRAY(0x7f78592ea520)},
                 url = {http://eprints.lincoln.ac.uk/24853/},
            abstract = {There is increasing interest in using Qualitative Spatial
    Relations as a formalism to abstract from noisy and
    large amounts of video data in order to form high level
    conceptualisations, e.g. of activities present in video.
    We present a library to support such work. It is compatible
    with the Robot Operating System (ROS) but can
    also be used stand alone. A number of QSRs are built
    in; others can be easily added.}
    }
  • K. Gerling, D. Hebesberger, C. Dondrup, T. K$backslash$"ortner, and M. Hanheide, “Robot deployment in long-term care: a case study of a mobile robot in physical therapy,” Zeitschrift für Geriatrie und Gerontologie, vol. 49, iss. 4, pp. 288-297, 2016.
    [BibTeX] [Abstract] [EPrints]

    Background. Healthcare systems in industrialised countries are challenged to provide care for a growing number of older adults. Information technology holds the promise of facilitating this process by providing support for care staff, and improving wellbeing of older adults through a variety of support systems. Goal. Little is known about the challenges that arise from the deployment of technology in care settings; yet, the integration of technology into care is one of the core determinants of successful support. In this paper, we discuss challenges and opportunities associated with technology integration in care using the example of a mobile robot to support physical therapy among older adults with cognitive impairment in the European project STRANDS. Results and discussion. We report on technical challenges along with perspectives of physical therapists, and provide an overview of lessons learned which we hope will help inform the work of researchers and practitioners wishing to integrate robotic aids in the caregiving process.

    @article{lirolem22902,
              volume = {49},
              number = {4},
               month = {June},
              author = {Kathrin Gerling and Denise Hebesberger and Christian Dondrup and Tobias K{$\backslash$}"ortner and Marc Hanheide},
               title = {Robot deployment in long-term care: a case study of a mobile robot in physical therapy},
           publisher = {Springer for Bundesverband Geriatrie / Deutsche Gesellschaft f{\"u}r Gerontologie und Geriatrie},
                year = {2016},
             journal = {Zeitschrift f{\"u}r Geriatrie und Gerontologie},
               pages = {288--297},
            keywords = {ARRAY(0x7f78592e4150)},
                 url = {http://eprints.lincoln.ac.uk/22902/},
            abstract = {Background. Healthcare systems in industrialised countries are challenged to provide
    care for a growing number of older adults. Information technology holds the promise of
    facilitating this process by providing support for care staff, and improving wellbeing of
    older adults through a variety of support systems. Goal. Little is known about the
    challenges that arise from the deployment of technology in care settings; yet, the
    integration of technology into care is one of the core determinants of successful
    support. In this paper, we discuss challenges and opportunities associated with
    technology integration in care using the example of a mobile robot to support physical
    therapy among older adults with cognitive impairment in the European project
    STRANDS. Results and discussion. We report on technical challenges along with
    perspectives of physical therapists, and provide an overview of lessons learned which
    we hope will help inform the work of researchers and practitioners wishing to integrate
    robotic aids in the caregiving process.}
    }
  • E. Gyebi, M. Hanheide, and G. Cielniak, “The effectiveness of integrating educational robotic activities into higher education Computer Science curricula: a case study in a developing country,” in Edurobotics 2016, 2016, pp. 73-87.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a case study to investigate the effects of educational robotics on a formal undergraduate Computer Science education in a developing country. The key contributions of this paper include a longitudinal study design, spanning the whole duration of one taught course, and its focus on continually assessing the effectiveness and the impact of robotic-based exercises. The study assessed the students’ motivation, engagement and level of understanding in learning general computer programming. The survey results indicate that there are benefits which can be gained from such activities and educational robotics is a promising tool in developing engaging study curricula. We hope that our experience from this study together with the free materials and data available for download will be beneficial to other practitioners working with educational robotics in different parts of the world.

    @inproceedings{lirolem25579,
           booktitle = {Edurobotics 2016},
               month = {November},
               title = {The effectiveness of integrating educational robotic activities into higher education Computer Science curricula: a case study in a developing country},
              author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
           publisher = {Springer},
                year = {2016},
               pages = {73--87},
            keywords = {ARRAY(0x7f78592da810)},
                 url = {http://eprints.lincoln.ac.uk/25579/},
            abstract = {In this paper, we present a case study to investigate the effects of educational robotics on a formal undergraduate Computer Science education in a developing country. The key contributions of this paper include a longitudinal study design, spanning the whole duration of one taught course, and its focus on continually assessing the effectiveness and the impact of robotic-based exercises. The study assessed the  students' motivation, engagement and level of understanding in learning general computer programming. The survey results indicate that there are benefits which can be gained from such activities and educational robotics is a promising tool in developing engaging study curricula. We hope that our experience from this study together with the free materials and data available for download will be beneficial to other practitioners working with educational robotics in different parts of the world.}
    }
  • C. Hu, F. Arvin, C. Xiong, and S. Yue, “A bio-inspired embedded vision system for autonomous micro-robots: the LGMD case,” IEEE Transactions on Cognitive and Developmental Systems, vol. PP, iss. 99, pp. 1-14, 2016.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a new bio-inspired vision system embedded for micro-robots. The vision system takes inspiration from locusts in detecting fast approaching objects. Neurophysiological research suggested that locusts use a wide-field visual neuron called lobula giant movement detector (LGMD) to respond to imminent collisions. In this work, we present the implementation of the selected neuron model by a low-cost ARM processor as part of a composite vision module. As the first embedded LGMD vision module fits to a micro-robot, the developed system performs all image acquisition and processing independently. The vision module is placed on top of a microrobot to initiate obstacle avoidance behaviour autonomously. Both simulation and real-world experiments were carried out to test the reliability and robustness of the vision system. The results of the experiments with different scenarios demonstrated the potential of the bio-inspired vision system as a low-cost embedded module for autonomous robots.

    @article{lirolem25279,
              volume = {PP},
              number = {99},
               month = {May},
              author = {Cheng Hu and Farshad Arvin and Caihua Xiong and Shigang Yue},
               title = {A bio-inspired embedded vision system for autonomous micro-robots: the LGMD case},
           publisher = {IEEE},
                year = {2016},
             journal = {IEEE Transactions on Cognitive and Developmental Systems},
               pages = {1--14},
            keywords = {ARRAY(0x7f78592ec8c8)},
                 url = {http://eprints.lincoln.ac.uk/25279/},
            abstract = {In this paper, we present a new bio-inspired vision
    system embedded for micro-robots. The vision system takes inspiration
    from locusts in detecting fast approaching objects. Neurophysiological
    research suggested that locusts use a wide-field
    visual neuron called lobula giant movement detector (LGMD)
    to respond to imminent collisions. In this work, we present
    the implementation of the selected neuron model by a low-cost
    ARM processor as part of a composite vision module. As the
    first embedded LGMD vision module fits to a micro-robot, the
    developed system performs all image acquisition and processing
    independently. The vision module is placed on top of a microrobot
    to initiate obstacle avoidance behaviour autonomously. Both
    simulation and real-world experiments were carried out to test
    the reliability and robustness of the vision system. The results
    of the experiments with different scenarios demonstrated the
    potential of the bio-inspired vision system as a low-cost embedded
    module for autonomous robots.}
    }
  • J. Kennedy, P. Baxter, E. Senft, and T. Belpaeme, “Social robot tutoring for child second language learning,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction HRI 2016, Christchurch, New Zealand, 2016, pp. 231-238.
    [BibTeX] [Abstract] [EPrints]

    An increasing amount of research is being conducted to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot?s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ?low? and ?high? verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot?s verbal behaviour may not return the desired positive impact on learning gains.

    @inproceedings{lirolem24855,
               month = {March},
              author = {James Kennedy and Paul Baxter and Emmanuel Senft and Tony Belpaeme},
           booktitle = {Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction HRI 2016},
             address = {Christchurch, New Zealand},
               title = {Social robot tutoring for child second language learning},
           publisher = {ACM Press},
               pages = {231--238},
                year = {2016},
            keywords = {ARRAY(0x7f78593a53f8)},
                 url = {http://eprints.lincoln.ac.uk/24855/},
            abstract = {An increasing amount of research is being conducted
    to determine how a robot tutor should behave socially in educa- tional interactions with children. Both human-human and human- robot interaction literature predicts an increase in learning with increased social availability of a tutor, where social availability has verbal and nonverbal components. Prior work has shown that greater availability in the nonverbal behaviour of a robot tutor has a positive impact on child learning. This paper presents a study with 67 children to explore how social aspects of a tutor robot?s speech influences their perception of the robot and their language learning in an interaction. Children perceive the difference in social behaviour between ?low? and ?high? verbal availability conditions, and improve significantly between a pre- and a post-test in both conditions. A longer-term retention test taken the following week showed that the children had retained almost all of the information they had learnt. However, learning was not affected by which of the robot behaviours they had been exposed to. It is suggested that in this short-term interaction context, additional effort in developing social aspects of a robot?s verbal behaviour may not return the desired positive impact on learning gains.}
    }
  • T. Krajnik, J. P. Fentanes, J. Santos, and T. Duckett, “Frequency map enhancement: introducing dynamics into static environment models,” in ICRA Workshop AI for Long-Term Autonomy, 2016.
    [BibTeX] [Abstract] [EPrints]

    We present applications of the Frequency Map Enhancement (FreMEn), which improves the performance of mobile robots in long-term scenarios by introducing the notion of dynamics into their (originally static) environment models. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by their frequency spectra. This allows to integrate sparse and irregular observations obtained during long-term deployments of mobile robots into memory-efficient spatio-temporal models that reflect mid- and long-term pseudo-periodic environment variations. The frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments. In a series of experiments performed over periods of weeks to years, we demonstrate that the proposed approach improves mobile robot localization, path and task planning, activity recognition and allows for life-long spatio-temporal exploration.

    @inproceedings{lirolem23261,
           booktitle = {ICRA Workshop AI for Long-Term Autonomy},
               month = {May},
               title = {Frequency map enhancement: introducing dynamics into static environment models},
              author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Tom Duckett},
                year = {2016},
            keywords = {ARRAY(0x7f78592ecca0)},
                 url = {http://eprints.lincoln.ac.uk/23261/},
            abstract = {We present applications of the Frequency Map Enhancement (FreMEn), which improves the performance of mobile robots in long-term scenarios by introducing the notion of dynamics into their (originally static) environment models. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by their frequency spectra. This allows to integrate sparse and irregular observations obtained during long-term deployments of mobile robots into memory-efficient spatio-temporal models that reflect mid- and long-term pseudo-periodic environment variations. The frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments.   In a series of experiments performed over periods of weeks to years, we demonstrate that the proposed approach improves mobile robot localization, path and task planning, activity recognition and allows for life-long spatio-temporal exploration.}
    }
  • M. Kulich, T. Krajnik, L. Preucil, and T. Duckett, “To explore or to exploit? Learning humans’ behaviour to maximize interactions with them,” in International Workshop on Modelling and Simulation for Autonomous Systems, 2016, pp. 48-63.
    [BibTeX] [Abstract] [EPrints]

    Assume a robot operating in a public space (e.g., a library, a museum) and serving visitors as a companion, a guide or an information stand. To do that, the robot has to interact with humans, which presumes that it actively searches for humans in order to interact with them. This paper addresses the problem how to plan robot’s actions in order to maximize the number of such interactions in the case human behavior is not known in advance. We formulate this problem as the exploration/exploitation problem and design several strategies for the robot. The main contribution of the paper than lies in evaluation and comparison of the designed strategies on two datasets. The evaluation shows interesting properties of the strategies, which are discussed.

    @inproceedings{lirolem26195,
           booktitle = {International Workshop on Modelling and Simulation for Autonomous Systems},
               month = {June},
               title = {To explore or to exploit? Learning humans' behaviour to maximize interactions with them},
              author = {Miroslav Kulich and Tomas Krajnik and Libor Preucil and Tom Duckett},
           publisher = {Springer},
                year = {2016},
               pages = {48--63},
            keywords = {ARRAY(0x7f78592e60d8)},
                 url = {http://eprints.lincoln.ac.uk/26195/},
            abstract = {Assume a robot operating in a public space (e.g., a library, a museum) and serving visitors as a companion, a guide or an information stand. To do  that, the robot has to interact with humans, which presumes that it actively searches for humans in order to interact with them. This paper addresses the problem how to plan robot's actions in order to maximize the number of such interactions in the case human behavior is not known in advance. We formulate this problem as the exploration/exploitation problem and design several strategies for the robot. The main contribution of the paper than lies in evaluation and comparison of the designed strategies on two datasets. The evaluation shows interesting properties of the strategies, which are discussed.}
    }
  • K. Kusumam, T. Krajnik, S. Pearson, G. Cielniak, and T. Duckett, “Can you pick a broccoli? 3D-vision based detection and localisation of broccoli heads in the field,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier and a temporal filter to track the detected heads results in a system that detects broccoli heads with 95.2\% precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field.

    @inproceedings{lirolem24087,
           booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
               month = {October},
               title = {Can you pick a broccoli? 3D-vision based detection and localisation of broccoli heads in the field},
              author = {Keerthy Kusumam and Tomas Krajnik and Simon Pearson and Grzegorz Cielniak and Tom Duckett},
           publisher = {IEEE},
                year = {2016},
            keywords = {ARRAY(0x7f78592c6260)},
                 url = {http://eprints.lincoln.ac.uk/24087/},
            abstract = {This paper presents a 3D vision system for robotic harvesting of broccoli using low-cost RGB-D sensors. The presented method addresses the tasks of detecting mature broccoli heads in the field and providing their 3D locations relative to the vehicle. The paper evaluates different 3D features, machine learning and temporal filtering methods for detection of broccoli heads. Our experiments show that a combination of Viewpoint Feature Histograms, Support Vector Machine classifier and a temporal filter to track the detected heads results in a system that detects broccoli heads with 95.2\% precision. We also show that the temporal filtering can be used to generate a 3D map of the broccoli head positions in the field.}
    }
  • X. Li, W. Qi, and S. Yue, “An effective pansharpening method based on guided filtering,” in 2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA), 2016, pp. 534-538.
    [BibTeX] [Abstract] [EPrints]

    Pansharpening is an important tool in remote sensing applications. It transforms a set of low-spatial-resolution multispectral images to high-spatial-resolution images by fusing with a co-registered high-spatial-resolution panchromatic image. To deal with the increasing high resolution satellite images, wide varieties of pansharpening techniques have been developed. In this paper, we present an effective pansharpening method based on guided filtering. The method takes advantage of the guided filter to refine the blocking edges in the upscaled multispectral images and extract sufficient high frequency details from the panchromatic image. Moreover, it can be implemented to sharpen multispectral imagery in a convenient band-by-band manner. The experimental evaluations are carried out on QuickBird satellite images. Subjective and objective evaluations show that our proposed method can achieve high spectral and spatial quality and outperforms some existing methods.

    @inproceedings{lirolem27952,
           booktitle = {2016 IEEE 11th Conference on Industrial Electronics and Applications (ICIEA)},
               month = {June},
               title = {An effective pansharpening method based on guided filtering},
              author = {Xu Li and Weifeng Qi and Shigang Yue},
                year = {2016},
               pages = {534--538},
            keywords = {ARRAY(0x7f78592e6888)},
                 url = {http://eprints.lincoln.ac.uk/27952/},
            abstract = {Pansharpening is an important tool in remote sensing applications. It transforms a set of low-spatial-resolution multispectral images to high-spatial-resolution images by fusing with a co-registered high-spatial-resolution panchromatic image. To deal with the increasing high resolution satellite images, wide varieties of pansharpening techniques have been developed. In this paper, we present an effective pansharpening method based on guided filtering. The method takes advantage of the guided filter to refine the blocking edges in the upscaled multispectral images and extract sufficient high frequency details from the panchromatic image. Moreover, it can be implemented to sharpen multispectral imagery in a convenient band-by-band manner. The experimental evaluations are carried out on QuickBird satellite images. Subjective and objective evaluations show that our proposed method can achieve high spectral and spatial quality and outperforms some existing methods.}
    }
  • F. Lier, M. Hanheide, L. Natale, S. Schulz, J. Weisz, S. Wachsmuth, and S. Wrede, “Towards automated system and experiment reproduction in robotics,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016.
    [BibTeX] [Abstract] [EPrints]

    Even though research on autonomous robots and human-robot interaction accomplished great progress in recent years, and reusable soft- and hardware components are available, many of the reported findings are only hardly reproducible by fellow scientists. Usually, reproducibility is impeded because required information, such as the specification of software versions and their configuration, required data sets, and experiment protocols are not mentioned or referenced in most publications. In order to address these issues, we recently introduced an integrated tool chain and its underlying development process to facilitate reproducibility in robotics. In this contribution we instantiate the complete tool chain in a unique user study in order to assess its applicability and usability. To this end, we chose three different robotic systems from independent institutions and modeled them in our tool chain, including three exemplary experiments. Subsequently, we asked twelve researchers to reproduce one of the formerly unknown systems and the associated experiment. We show that all twelve scientists were able to replicate a formerly unknown robotics experiment using our tool chain.

    @inproceedings{lirolem24852,
           booktitle = {2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
               month = {October},
               title = {Towards automated system and experiment reproduction in robotics},
              author = {Florian Lier and Marc Hanheide and Lorenzo Natale and Simon Schulz and Jonathan Weisz and Sven Wachsmuth and Sebastian Wrede},
                year = {2016},
            keywords = {ARRAY(0x7f78592b77b8)},
                 url = {http://eprints.lincoln.ac.uk/24852/},
            abstract = {Even though research on autonomous robots and
    human-robot interaction accomplished great progress in recent
    years, and reusable soft- and hardware components are
    available, many of the reported findings are only hardly
    reproducible by fellow scientists. Usually, reproducibility is
    impeded because required information, such as the specification
    of software versions and their configuration, required data sets,
    and experiment protocols are not mentioned or referenced
    in most publications. In order to address these issues, we
    recently introduced an integrated tool chain and its underlying
    development process to facilitate reproducibility in robotics.
    In this contribution we instantiate the complete tool chain in
    a unique user study in order to assess its applicability and
    usability. To this end, we chose three different robotic systems
    from independent institutions and modeled them in our tool
    chain, including three exemplary experiments. Subsequently,
    we asked twelve researchers to reproduce one of the formerly
    unknown systems and the associated experiment. We show that
    all twelve scientists were able to replicate a formerly unknown
    robotics experiment using our tool chain.}
    }
  • D. Liu and S. Yue, “Visual pattern recognition using unsupervised spike timing dependent plasticity learning,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 285-292.
    [BibTeX] [Abstract] [EPrints]

    Neuroscience study shows mammalian brain only use millisecond scale time window to process complicated real-life recognition scenarios. However, such speed cannot be achieved by traditional rate-based spiking neural network (SNN). Compared with spiking rate, the specific spiking timing (also called spiking pattern) may convey much more information. In this paper, by using modified rank order coding scheme, the generated absolute analog features have been encoded into the first spike wave with specific spatiotemporal structural information. An intuitive yet powerful feed-forward spiking neural network framework has been proposed, along with its own unsupervised spike-timing-dependent plasticity (STDP) learning rule with dynamic post-synaptic potential threshold. Compared with other state-of-art spiking algorithms, the proposed method uses biologically plausible STDP learning method to learn the selectivity while the dynamic post-synaptic potential threshold guarantees no training sample will be ignored during the learning procedure. Furthermore, unlike the complicated frameworks used in those state-of-art spiking algorithms, the proposed intuitive spiking neural network is not time-consuming and quite capable of on-line learning. A satisfactory experimental result has been achieved on classic MNIST handwritten character database.

    @inproceedings{lirolem27954,
           booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
               month = {July},
               title = {Visual pattern recognition using unsupervised spike timing dependent plasticity learning},
              author = {Daqi Liu and Shigang Yue},
                year = {2016},
               pages = {285--292},
            keywords = {ARRAY(0x7f78592d54e0)},
                 url = {http://eprints.lincoln.ac.uk/27954/},
            abstract = {Neuroscience study shows mammalian brain only use millisecond scale time window to process complicated real-life recognition scenarios. However, such speed cannot be achieved by traditional rate-based spiking neural network (SNN). Compared with spiking rate, the specific spiking timing (also called spiking pattern) may convey much more information. In this paper, by using modified rank order coding scheme, the generated absolute analog features have been encoded into the first spike wave with specific spatiotemporal structural information. An intuitive yet powerful feed-forward spiking neural network framework has been proposed, along with its own unsupervised spike-timing-dependent plasticity (STDP) learning rule with dynamic post-synaptic potential threshold. Compared with other state-of-art spiking algorithms, the proposed method uses biologically plausible STDP learning method to learn the selectivity while the dynamic post-synaptic potential threshold guarantees no training sample will be ignored during the learning procedure. Furthermore, unlike the complicated frameworks used in those state-of-art spiking algorithms, the proposed intuitive spiking neural network is not time-consuming and quite capable of on-line learning. A satisfactory experimental result has been achieved on classic MNIST handwritten character database.}
    }
  • V. Modugno, G. Neumann, E. Rueckert, G. Oriolo, J. Peters, and S. Ivaldi, “Learning soft task priorities for control of redundant robots,” in IEEE International Conference on Robotics and Automation (ICRA) 2016, 2016.
    [BibTeX] [Abstract] [EPrints]

    Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is precoded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.

    @inproceedings{lirolem25639,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA) 2016},
               month = {May},
               title = {Learning soft task priorities for control of redundant robots},
              author = {V. Modugno and Gerhard Neumann and E. Rueckert and G. Oriolo and J. Peters and S. Ivaldi},
                year = {2016},
            keywords = {ARRAY(0x7f78593de350)},
                 url = {http://eprints.lincoln.ac.uk/25639/},
            abstract = {Movement primitives (MPs) provide a powerful
    framework for data driven movement generation that has been
    successfully applied for learning from demonstrations and robot
    reinforcement learning. In robotics we often want to solve a
    multitude of different, but related tasks. As the parameters
    of the primitives are typically high dimensional, a common
    practice for the generalization of movement primitives to new
    tasks is to adapt only a small set of control variables, also
    called meta parameters, of the primitive. Yet, for most MP
    representations, the encoding of these control variables is precoded
    in the representation and can not be adapted to the
    considered tasks. In this paper, we want to learn the encoding of
    task-specific control variables also from data instead of relying
    on fixed meta-parameter representations. We use hierarchical
    Bayesian models (HBMs) to estimate a low dimensional latent
    variable model for probabilistic movement primitives (ProMPs),
    which is a recent movement primitive representation. We show
    on two real robot datasets that ProMPs based on HBMs
    outperform standard ProMPs in terms of generalization and
    learning from a small amount of data and also allows for an
    intuitive analysis of the movement. We also extend our HBM by
    a mixture model, such that we can model different movement
    types in the same dataset.}
    }
  • T. Osa, J. Peters, and G. Neumann, “Experiments with hierarchical reinforcement learning of multiple grasping policies,” in Proceedings of the International Symposium on Experimental Robotics (ISER), 2016.
    [BibTeX] [Abstract] [EPrints]

    Robotic grasping has attracted considerable interest, but it still remains a challenging task. The data-driven approach is a promising solution to the robotic grasping problem; this approach leverages a grasp dataset and generalizes grasps for various objects. However, these methods often depend on the quality of the given datasets, which are not trivial to obtain with sufficient quality. Although reinforcement learning approaches have been recently used to achieve autonomous collection of grasp datasets, the existing algorithms are often limited to specific grasp types. In this paper, we present a framework for hierarchical reinforcement learning of grasping policies. In our framework, the lowerlevel hierarchy learns multiple grasp types, and the upper-level hierarchy learns a policy to select from the learned grasp types according to a point cloud of a new object. Through experiments, we validate that our approach learns grasping by constructing the grasp dataset autonomously. The experimental results show that our approach learns multiple grasping policies and generalizes the learned grasps by using local point cloud information.

    @inproceedings{lirolem26735,
           booktitle = {Proceedings of the International Symposium on Experimental Robotics (ISER)},
               month = {April},
               title = {Experiments with hierarchical reinforcement learning of multiple grasping policies},
              author = {T. Osa and J. Peters and G. Neumann},
                year = {2016},
            keywords = {ARRAY(0x7f78592b7338)},
                 url = {http://eprints.lincoln.ac.uk/26735/},
            abstract = {Robotic grasping has attracted considerable interest, but it
    still remains a challenging task. The data-driven approach is a promising
    solution to the robotic grasping problem; this approach leverages a
    grasp dataset and generalizes grasps for various objects. However, these
    methods often depend on the quality of the given datasets, which are not
    trivial to obtain with sufficient quality. Although reinforcement learning
    approaches have been recently used to achieve autonomous collection
    of grasp datasets, the existing algorithms are often limited to specific
    grasp types. In this paper, we present a framework for hierarchical reinforcement
    learning of grasping policies. In our framework, the lowerlevel
    hierarchy learns multiple grasp types, and the upper-level hierarchy
    learns a policy to select from the learned grasp types according to a point
    cloud of a new object. Through experiments, we validate that our approach
    learns grasping by constructing the grasp dataset autonomously.
    The experimental results show that our approach learns multiple grasping
    policies and generalizes the learned grasps by using local point cloud
    information.}
    }
  • W. Qi, X. Li, and S. Yue, “A guided filtering and HCT integrated pansharpening method for WorldView-2 satellite images,” in 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS), 2016, pp. 7272-7275.
    [BibTeX] [Abstract] [EPrints]

    Pansharpening has been an important tool in remote sensing field, which is a process of providing multispectral images with higher spatial resolution. When dealing with WorldView-2 satellite imagery having more bands and higher resolution, most existing methods are not effective. In this paper, we propose a novel and effective pansharpening methods combing guided filtering and hyperspherical color transformation (HCT) for WorldView-2 images. We use panchromatic image as the guidance to further refine the intensity of multispectral data and also to extract the sufficient details from the panchromatic image itself. Moreover, the guided filtering and HCT integrated scheme can inject the extracted details into the multispectral data and the multispectral images can be sharpened all at once with an arbitrary order. The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.

    @inproceedings{lirolem27953,
           booktitle = {2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS)},
               month = {July},
               title = {A guided filtering and HCT integrated pansharpening method for WorldView-2 satellite images},
              author = {Weifeng Qi and Xu Li and Shigang Yue},
                year = {2016},
               pages = {7272--7275},
            keywords = {ARRAY(0x7f78592c60f8)},
                 url = {http://eprints.lincoln.ac.uk/27953/},
            abstract = {Pansharpening has been an important tool in remote sensing field, which is a process of providing multispectral images with higher spatial resolution. When dealing with WorldView-2 satellite imagery having more bands and higher resolution, most existing methods are not effective. In this paper, we propose a novel and effective pansharpening methods combing guided filtering and hyperspherical color transformation (HCT) for WorldView-2 images. We use panchromatic image as the guidance to further refine the intensity of multispectral data and also to extract the sufficient details from the panchromatic image itself. Moreover, the guided filtering and HCT integrated scheme can inject the extracted details into the multispectral data and the multispectral images can be sharpened all at once with an arbitrary order. The experimental results show that our proposed method can obtain high-quality pansharpened results and outperforms some existing methods.}
    }
  • F. Riccio, R. Capobianco, M. Hanheide, and D. Nardi, “Stam: a framework for spatio-temporal affordance maps,” in International Workshop on Modelling and Simulation for Autonomous Systems, 2016, pp. 271-280.
    [BibTeX] [Abstract] [EPrints]

    A?ordances have been introduced in literature as action op- portunities that objects o?er, and used in robotics to semantically rep- resent their interconnection. However, when considering an environment instead of an object, the problem becomes more complex due to the dynamism of its state. To tackle this issue, we introduce the concept of Spatio-Temporal A?ordances (STA) and Spatio-Temporal A?ordance Map (STAM). Using this formalism, we encode action semantics re- lated to the environment to improve task execution capabilities of an autonomous robot. We experimentally validate our approach to support the execution of robot tasks by showing that a?ordances encode accurate semantics of the environment.

    @inproceedings{lirolem24851,
           booktitle = {International Workshop on Modelling and Simulation for Autonomous Systems},
               month = {June},
               title = {Stam: a framework for spatio-temporal affordance maps},
              author = {Francesco Riccio and Roberto Capobianco and Marc Hanheide and Daniele Nardi},
           publisher = {Springer},
                year = {2016},
               pages = {271--280},
            keywords = {ARRAY(0x7f78592c5fa8)},
                 url = {http://eprints.lincoln.ac.uk/24851/},
            abstract = {A?ordances have been introduced in literature as action op-
    portunities that objects o?er, and used in robotics to semantically rep-
    resent their interconnection. However, when considering an environment
    instead of an object, the problem becomes more complex due to the
    dynamism of its state. To tackle this issue, we introduce the concept
    of Spatio-Temporal A?ordances (STA) and Spatio-Temporal A?ordance
    Map (STAM). Using this formalism, we encode action semantics re-
    lated to the environment to improve task execution capabilities of an
    autonomous robot. We experimentally validate our approach to support
    the execution of robot tasks by showing that a?ordances encode accurate
    semantics of the environment.}
    }
  • C. Salatino, V. Gower, M. Ghrissi, A. Tapus, K. Wieczorowska-Tobis, A. Suwalska, P. Barattini, R. Rosso, G. Munaro, N. Bellotto, and H. van den Heuvel, “EnrichMe: a robotic solution for independence and active aging of elderly people with MCI,” in 15th International Conference on Computers Helping People with Special Needs (ICCHP 2016), 2016.
    [BibTeX] [Abstract] [EPrints]

    Mild cognitive impairment (MCI) is a state related to ageing, and sometimes evolves to dementia. As there is no pharmacological treatment for MCI, a non-pharmacological approach is very important. The use of Information and Communication Technologies (ICT) in care and assistance services for elderly people increases their chances of prolonging independence thanks to better cognitive efficiency. Robots are seen to have the potential to support the care and independence of elderly people. The project ENRICHME (funded by the EU H2020 Programme) focuses on developing and testing technologies for supporting elderly people with MCI in their living environment for a long time. This paper describes the results of the activities conducted during the first year of the ENRICHME project, in particular the definition of user needs and requirements and the resulting system architecture.

    @inproceedings{lirolem22704,
           booktitle = {15th International Conference on Computers Helping People with Special Needs (ICCHP 2016)},
               month = {July},
               title = {EnrichMe: a robotic solution for independence and active aging of elderly people with MCI},
              author = {Claudia Salatino and Valerio Gower and Meftah Ghrissi and Adriana Tapus and K Wieczorowska-Tobis and A Suwalska and Paolo Barattini and Roberto Rosso and Giulia Munaro and Nicola Bellotto and Herjan van den Heuvel},
                year = {2016},
            keywords = {ARRAY(0x7f78593e4808)},
                 url = {http://eprints.lincoln.ac.uk/22704/},
            abstract = {Mild cognitive impairment (MCI) is a state related to ageing, and sometimes evolves to dementia. As there is no pharmacological treatment for MCI, a non-pharmacological approach is very important. The use of Information and Communication Technologies (ICT) in care and assistance services for elderly people increases their chances of prolonging independence thanks to better cognitive efficiency. Robots are seen to have the potential to support the care and independence of elderly people. The project ENRICHME (funded by the EU H2020 Programme) focuses on developing and testing technologies for supporting elderly people with MCI in their living environment for a long time. This paper describes the results of the activities conducted during the first year of the ENRICHME project, in particular the definition of user needs and requirements and the resulting system architecture.}
    }
  • J. M. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, “Lifelong information-driven exploration to complete and refine 4-D spatio-temporal maps,” IEEE Robotics and Automation Letters, vol. 1, iss. 2, pp. 684-691, 2016.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an exploration method that allows mobile robots to build and maintain spatio-temporal models of changing environments. The assumption of a perpetuallychanging world adds a temporal dimension to the exploration problem, making spatio-temporal exploration a never-ending, life-long learning process. We address the problem by application of information-theoretic exploration methods to spatio-temporal models that represent the uncertainty of environment states as probabilistic functions of time. This allows to predict the potential information gain to be obtained by observing a particular area at a given time, and consequently, to decide which locations to visit and the best times to go there. To validate the approach, a mobile robot was deployed continuously over 5 consecutive business days in a busy office environment. The results indicate that the robot?s ability to spot environmental changes im

    @article{lirolem22698,
              volume = {1},
              number = {2},
               month = {July},
              author = {Joao Machado Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
               title = {Lifelong information-driven exploration to complete and refine 4-D spatio-temporal maps},
           publisher = {IEEE},
                year = {2016},
             journal = {IEEE Robotics and Automation Letters},
               pages = {684--691},
            keywords = {ARRAY(0x7f78592c2bf8)},
                 url = {http://eprints.lincoln.ac.uk/22698/},
            abstract = {This paper presents an exploration method that allows
    mobile robots to build and maintain spatio-temporal models
    of changing environments. The assumption of a perpetuallychanging
    world adds a temporal dimension to the exploration
    problem, making spatio-temporal exploration a never-ending,
    life-long learning process. We address the problem by application
    of information-theoretic exploration methods to spatio-temporal
    models that represent the uncertainty of environment states as
    probabilistic functions of time. This allows to predict the potential
    information gain to be obtained by observing a particular area
    at a given time, and consequently, to decide which locations to
    visit and the best times to go there.
    To validate the approach, a mobile robot was deployed
    continuously over 5 consecutive business days in a busy office
    environment. The results indicate that the robot?s ability to spot
    environmental changes im}
    }
  • J. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, “A 3D simulation environment with real dynamics: a tool for benchmarking mobile robot performance in long-term deployments,” in ICRA 2016 Workshop: AI for Long-term Autonomy, 2016.
    [BibTeX] [Abstract] [EPrints]

    This paper describes a method to compare and evaluate mobile robot algorithms for long-term deployment in changing environments. Typically, the long-term performance of state estimation algorithms for mobile robots is evaluated using pre-recorded sensory datasets. However such datasets are not suitable for evaluating decision-making and control algorithms where the behaviour of the robot will be different in every trial. Simulation allows to overcome this issue and while it ensures repeatability of experiments, the development of 3D simulations for an extended period of time is a costly exercise. In our approach long-term datasets comprising high-level tracks of dynamic entities such as people and furniture are recorded by ambient sensors placed in a real environment. The high-level tracks are then used to parameterise a 3D simulation containing its own geometric models of the dynamic entities and the background scene. This simulation, which is based on actual human activities, can then be used to benchmark and validate algorithms for long-term operation of mobile robots.

    @inproceedings{lirolem23220,
           booktitle = {ICRA 2016 Workshop: AI for Long-term Autonomy},
               month = {May},
               title = {A 3D simulation environment with real dynamics: a tool for benchmarking mobile robot performance in long-term deployments},
              author = {Joao Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
                year = {2016},
            keywords = {ARRAY(0x7f78592ee118)},
                 url = {http://eprints.lincoln.ac.uk/23220/},
            abstract = {This paper describes a method to compare and evaluate mobile robot algorithms for long-term deployment in changing   environments. Typically, the long-term performance of state estimation algorithms for mobile robots is evaluated using pre-recorded sensory datasets. However such datasets are not suitable for evaluating decision-making and control  algorithms where the behaviour of the robot will be different in every trial. Simulation allows to overcome this issue and while it ensures repeatability of experiments, the development of 3D simulations for an extended period of time is a costly exercise.
    In our approach long-term datasets comprising high-level tracks of dynamic entities such as people and furniture are recorded by ambient sensors placed in a real environment. The high-level tracks are then used to parameterise a 3D  simulation containing its own geometric models of the dynamic entities and the background scene. This simulation,  which is based on actual human activities, can then be used to benchmark and validate algorithms for long-term  operation of mobile robots.}
    }
  • D. Skovcaj, A. Vrevcko, M. Mahnivc, M. Janívcek, G. M. Kruijff, M. Hanheide, N. Hawes, J. L. Wyatt, T. Keller, K. Zhou, M. Zillich, and M. Kristan, “An integrated system for interactive continuous learning of categorical knowledge,” Journal of Experimental & Theoretical Artificial Intelligence, vol. 28, iss. 5, pp. 823-848, 2016.
    [BibTeX] [Abstract] [EPrints]

    This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.

    @article{lirolem22203,
              volume = {28},
              number = {5},
               month = {August},
              author = {Danijel Sko{\vc}aj and Alen Vre{\vc}ko and Marko Mahni{\vc} and Miroslav Jan{\'i}{\vc}ek and Geert-Jan M Kruijff and Marc Hanheide and Nick Hawes and Jeremy L Wyatt and Thomas Keller and Kai Zhou and Michael Zillich and Matej Kristan},
               title = {An integrated system for interactive continuous learning of categorical knowledge},
           publisher = {Taylor \& Francis: STM, Behavioural Science and Public Health Titles},
                year = {2016},
             journal = {Journal of Experimental \& Theoretical Artificial Intelligence},
               pages = {823--848},
            keywords = {ARRAY(0x7f78592dd788)},
                 url = {http://eprints.lincoln.ac.uk/22203/},
            abstract = {This article presents an integrated robot system capable of interactive learning in dialogue with a human. Such a system needs to have several competencies and must be able to process different types of representations. In this article, we describe a collection of mechanisms that enable integration of heterogeneous competencies in a principled way. Central to our design is the creation of beliefs from visual and linguistic information, and the use of these beliefs for planning system behaviour to satisfy internal drives. The system is able to detect gaps in its knowledge and to plan and execute actions that provide information needed to fill these gaps. We propose a hierarchy of mechanisms which are capable of engaging in different kinds of learning interactions, e.g. those initiated by a tutor or by the system itself. We present the theory these mechanisms are build upon and an instantiation of this theory in the form of an integrated robot system. We demonstrate the operation of the system in the case of learning conceptual models of objects and their visual properties.}
    }
  • H. Wang, J. Peng, and S. Yue, “Bio-inspired small target motion detector with a new lateral inhibition mechanism,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 4751-4758.
    [BibTeX] [Abstract] [EPrints]

    In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.

    @inproceedings{lirolem27956,
           booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
               month = {July},
               title = {Bio-inspired small target motion detector with a new lateral inhibition mechanism},
              author = {Hongxin Wang and Jigen Peng and Shigang Yue},
                year = {2016},
               pages = {4751--4758},
            keywords = {ARRAY(0x7f78592ea8f8)},
                 url = {http://eprints.lincoln.ac.uk/27956/},
            abstract = {In nature, it is an important task for animals to detect small targets which move within cluttered background. In recent years, biologists have found that a class of neurons in the lobula complex, called STMDs (small target motion detectors) which have extreme selectivity for small targets moving within visual clutter. At the same time, some researchers assert that lateral inhibition plays an important role in discriminating the motion of the target from the motion of the background, even account for many features of the tuning of higher order visual neurons. Inspired by the finding that complete lateral inhibition can only be seen when the motion of the central region is identical to the motion of the peripheral region, we propose a new lateral inhibition mechanism combined with motion velocity and direction to improve the performance of ESTMD model (elementary small target motion detector). In this paper, we will elaborate on the biological plausibility and functionality of this new lateral inhibition mechanism in small target motion detection.}
    }
  • C. Wirth, J. Furnkranz, and G. Neumann, “Model-free preference-based reinforcement learning,” in Thirtieth AAAI Conference on Artificial Intelligence, 2016, pp. 2222-2228.
    [BibTeX] [Abstract] [EPrints]

    Specifying a numeric reward function for reinforcement learning typically requires a lot of hand-tuning from a human expert. In contrast, preference-based reinforcement learning (PBRL) utilizes only pairwise comparisons between trajectories as a feedback signal, which are often more intuitive to specify. Currently available approaches to PBRL for control problems with continuous state/action spaces require a known or estimated model, which is often not available and hard to learn. In this paper, we integrate preference-based estimation of the reward function into a model-free reinforcement learning (RL) algorithm, resulting in a model-free PBRL algorithm. Our new algorithm is based on Relative Entropy Policy Search (REPS), enabling us to utilize stochastic policies and to directly control the greediness of the policy update. REPS decreases exploration of the policy slowly by limiting the relative entropy of the policy update, which ensures that the algorithm is provided with a versatile set of trajectories, and consequently with informative preferences. The preference-based estimation is computed using a sample-based Bayesian method, which can also estimate the uncertainty of the utility. Additionally, we also compare to a linear solvable approximation, based on inverse RL. We show that both approaches perform favourably to the current state-of-the-art. The overall result is an algorithm that can learn non-parametric continuous action policies from a small number of preferences.

    @inproceedings{lirolem25746,
           booktitle = {Thirtieth AAAI Conference on Artificial Intelligence},
               month = {February},
               title = {Model-free preference-based reinforcement learning},
              author = {C. Wirth and J. Furnkranz and G. Neumann},
                year = {2016},
               pages = {2222--2228},
             journal = {30th AAAI Conference on Artificial Intelligence, AAAI 2016},
            keywords = {ARRAY(0x7f78592befe0)},
                 url = {http://eprints.lincoln.ac.uk/25746/},
            abstract = {Specifying a numeric reward function for reinforcement learning typically requires a lot of hand-tuning from a human expert. In contrast, preference-based reinforcement learning (PBRL) utilizes only pairwise comparisons between trajectories as a feedback signal, which are often more intuitive to specify. Currently available approaches to PBRL for control problems with continuous state/action spaces require a known or estimated model, which is often not available and hard to learn. In this paper, we integrate preference-based estimation of the reward function into a model-free reinforcement learning (RL) algorithm, resulting in a model-free PBRL algorithm. Our new algorithm is based on Relative Entropy Policy Search (REPS), enabling us to utilize stochastic policies and to directly control the greediness of the policy update. REPS decreases exploration of the policy slowly by limiting the relative entropy of the policy update, which ensures that the algorithm is provided with a versatile set of trajectories, and consequently with informative preferences. The preference-based estimation is computed using a sample-based Bayesian method, which can also estimate the uncertainty of the utility. Additionally, we also compare to a linear solvable approximation, based on inverse RL. We show that both approaches perform favourably to the current state-of-the-art. The overall result is an algorithm that can learn non-parametric continuous action policies from a small number of preferences.}
    }
  • C. Xiong, W. Chen, B. Sun, M. Liu, S. Yue, and W. Chen, “Design and implementation of an anthropomorphic hand for replicating human grasping functions,” IEEE Transactions on Robotics, vol. 32, iss. 3, pp. 652-671, 2016.
    [BibTeX] [Abstract] [EPrints]

    How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand. Â\copyright 2016 IEEE.

    @article{lirolem23735,
              volume = {32},
              number = {3},
               month = {June},
              author = {Cai-Hua Xiong and Wen-Rui Chen and Bai-Yang Sun and Ming-Jin Liu and Shigang Yue and Wen-Bin Chen},
               title = {Design and implementation of an anthropomorphic hand for replicating human grasping functions},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
                year = {2016},
             journal = {IEEE Transactions on Robotics},
               pages = {652--671},
            keywords = {ARRAY(0x7f78592c6020)},
                 url = {http://eprints.lincoln.ac.uk/23735/},
            abstract = {How to design an anthropomorphic hand with a few actuators to replicate the grasping functions of the human hand is still a challenging problem. This paper aims to develop a general theory for designing the anthropomorphic hand and endowing the designed hand with natural grasping functions. A grasping experimental paradigm was set up for analyzing the grasping mechanism of the human hand in daily living. The movement relationship among joints in a digit, among digits in the human hand, and the postural synergic characteristic of the fingers were studied during the grasping. The design principle of the anthropomorphic mechanical digit that can reproduce the digit grasping movement of the human hand was developed. The design theory of the kinematic transmission mechanism that can be embedded into the palm of the anthropomorphic hand to reproduce the postural synergic characteristic of the fingers by using a limited number of actuators is proposed. The design method of the anthropomorphic hand for replicating human grasping functions was formulated. Grasping experiments are given to verify the effectiveness of the proposed design method of the anthropomorphic hand. {\^A}{\copyright} 2016 IEEE.}
    }
  • Z. Xuqiang, L. Fule, W. Zhijun, L. Weitao, J. Wen, W. Zhihua, and S. Yue, “An S/H circuit with parasitics optimized for IF-sampling,” Journal of Semiconductors, vol. 37, iss. 6, p. 65005, 2016.
    [BibTeX] [Abstract] [EPrints]

    An IF-sampling S/H is presented, which adopts a flip-around structure, bottom-plate sampling technique and improved input bootstrapped switches. To achieve high sampling linearity over a wide input frequency range, the floating well technique is utilized to optimize the input switches. Besides, techniques of transistor load linearization and layout improvement are proposed to further reduce and linearize the parasitic capacitance. The S/H circuit has been fabricated in 0.18-\ensuremath\mum CMOS process as the front-end of a 14 bit, 250 MS/s pipeline ADC. For 30 MHz input, the measured SFDR/SNDR of the ADC is 94.7 dB/68. 5dB, which can remain over 84.3 dB/65.4 dB for input frequency up to 400 MHz. The ADC presents excellent dynamic performance at high input frequency, which is mainly attributed to the parasitics optimized S/H circuit.

    @article{lirolem27940,
              volume = {37},
              number = {6},
               month = {June},
              author = {Zheng Xuqiang and Li Fule and Wang Zhijun and Li Weitao and Jia Wen and Wang Zhihua and Shigang Yue},
               title = {An S/H circuit with parasitics optimized for IF-sampling},
           publisher = {IOP Publishing / Chinese Institute of Electronics},
                year = {2016},
             journal = {Journal of Semiconductors},
               pages = {065005},
            keywords = {ARRAY(0x7f78592ea148)},
                 url = {http://eprints.lincoln.ac.uk/27940/},
            abstract = {An IF-sampling S/H is presented, which adopts a flip-around structure, bottom-plate sampling technique and improved input bootstrapped switches. To achieve high sampling linearity over a wide input frequency range, the floating well technique is utilized to optimize the input switches. Besides, techniques of transistor load linearization and layout improvement are proposed to further reduce and linearize the parasitic capacitance. The S/H circuit has been fabricated in 0.18-{\ensuremath{\mu}}m CMOS process as the front-end of a 14 bit, 250 MS/s pipeline ADC. For 30 MHz input, the measured SFDR/SNDR of the ADC is 94.7 dB/68. 5dB, which can remain over 84.3 dB/65.4 dB for input frequency up to 400 MHz. The ADC presents excellent dynamic performance at high input frequency, which is mainly attributed to the parasitics optimized S/H circuit.}
    }
  • Y. Yang, A. Ahmed, S. Yue, X. Xie, H. Chen, and Z. Wang, “An algorithm for accurate needle orientation,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 2016, pp. 5095-5098.
    [BibTeX] [Abstract] [EPrints]

    For the early diagnosis and treatment, a needle insertion for biopsy and treatment is a common and important means. To solve the low accuracy and high probability of repeat surgery in traditional surgical procedures, a computer-assisted system is an effective solution. In such a system, how to acquire the accurate orientation of the surgical needle is one of the most important factors. This paper proposes a ?Center Point Method? for needle axis extraction with high accuracy. The method makes full use of edge points from two sides of the needle in image and creates center points through which an accurate axis is extracted. Experiments show that the proposed method improves needle orientation accuracy by approximately 70\% compared to related work in binocular stereovision system.

    @inproceedings{lirolem27949,
           booktitle = {2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
               month = {August},
               title = {An algorithm for accurate needle orientation},
              author = {Yifan Yang and Amr Ahmed and Shigang Yue and Xiang Xie and Hong Chen and Zhihua Wang},
                year = {2016},
               pages = {5095--5098},
            keywords = {ARRAY(0x7f78593e4f70)},
                 url = {http://eprints.lincoln.ac.uk/27949/},
            abstract = {For the early diagnosis and treatment, a needle insertion for biopsy and treatment is a common and important means. To solve the low accuracy and high probability of repeat surgery in traditional surgical procedures, a computer-assisted system is an effective solution. In such a system, how to acquire the accurate orientation of the surgical needle is one of the most important factors. This paper proposes a ?Center Point Method? for needle axis extraction with high accuracy. The method makes full use of edge points from two sides of the needle in image and creates center points through which an accurate axis is extracted. Experiments show that the proposed method improves needle orientation accuracy by approximately 70\% compared to related work in binocular stereovision system.}
    }
  • G. Zhang, C. Zhang, and S. Yue, “LGMD and DSNs neural networks integration for collision predication,” in 2016 International Joint Conference on Neural Networks (IJCNN), 2016, pp. 1174-1179.
    [BibTeX] [Abstract] [EPrints]

    An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.

    @inproceedings{lirolem27955,
           booktitle = {2016 International Joint Conference on Neural Networks (IJCNN)},
               month = {July},
               title = {LGMD and DSNs neural networks integration for collision predication},
              author = {Guopeng Zhang and Chun Zhang and Shigang Yue},
           publisher = {IEEE},
                year = {2016},
               pages = {1174--1179},
            keywords = {ARRAY(0x7f78593e42e0)},
                 url = {http://eprints.lincoln.ac.uk/27955/},
            abstract = {An ability to predict collisions is essential for current vehicles and autonomous robots. In this paper, an integrated collision predication system is proposed based on neural subsystems inspired from Lobula giant movement detector (LGMD) and directional selective neurons (DSNs) which focus on different part of the visual field separately. The two type of neurons found in the visual pathways of insects respond most strongly to moving objects with preferred motion patterns, i.e., the LGMD prefers looming stimuli and DSNs prefer specific lateral movements. We fuse the extracted information by each type of neurons to make final decision. By dividing the whole field of view into four regions for each subsystem to process, the proposed approaches can detect hazardous situations that had been difficult for single subsystem only. Our experiments show that the integrated system works in most of the hazardous scenarios.}
    }
  • X. Zheng, C. Zhang, F. Lv, F. Zhao, S. Yue, Z. Wang, F. Li, and Z. Wang, “A 5-50 Gb/s quarter rate transmitter with a 4-tap multiple-MUX based FFE in 65 nm CMOS,” in ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference, 2016, pp. 305-308.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a 5-50 Gb/s quarter-rate transmitter with a 4-tap feed-forward equalization (FFE) based on multiple-multiplexer (MUX). A bandwidth enhanced 4:1 MUX with the capability of eliminating charge-sharing effect is proposed to increase the maximum operating speed. To produce the quarter-rate parallel data streams with appropriate delays, a compact latch array associated with an interleaved-retiming technique is designed. Implemented in 65 nm CMOS technology, the transmitter occupying an area of 0.6 mm2 achieves a maximum data rate of 50 Gb/s with an energy efficiency of 3.1 pJ/bit.

    @inproceedings{lirolem27950,
           booktitle = {ESSCIRC Conference 2016: 42nd European Solid-State Circuits Conference},
               month = {September},
               title = {A 5-50 Gb/s quarter rate transmitter with a 4-tap multiple-MUX based FFE in 65 nm CMOS},
              author = {Xuqiang Zheng and Chun Zhang and Fangxu Lv and Feng Zhao and Shigang Yue and Ziqiang Wang and Fule Li and Zhihua Wang},
                year = {2016},
               pages = {305--308},
            keywords = {ARRAY(0x7f78592d9a48)},
                 url = {http://eprints.lincoln.ac.uk/27950/},
            abstract = {This paper presents a 5-50 Gb/s quarter-rate transmitter with a 4-tap feed-forward equalization (FFE) based on multiple-multiplexer (MUX). A bandwidth enhanced 4:1 MUX with the capability of eliminating charge-sharing effect is proposed to increase the maximum operating speed. To produce the quarter-rate parallel data streams with appropriate delays, a compact latch array associated with an interleaved-retiming technique is designed. Implemented in 65 nm CMOS technology, the transmitter occupying an area of 0.6 mm2 achieves a maximum data rate of 50 Gb/s with an energy efficiency of 3.1 pJ/bit.}
    }
  • X. Zheng, Z. Wang, F. Li, F. Zhao, S. Yue, C. Zhang, and Z. Wang, “A 14-bit 250 MS/s IF sampling pipelined ADC in 180 nm CMOS process,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 63, iss. 9, pp. 1381-1392, 2016.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a 14-bit 250 MS/s ADC fabricated in a 180 nm CMOS process, which aims at optimizing its linearity, operating speed, and power efficiency. The implemented ADC employs an improved SHA with parasitic optimized bootstrapped switches to achieve high sampling linearity over a wide input frequency range. It also explores a dedicated foreground calibration to correct the capacitor mismatches and the gain error of residue amplifier, where a novel configuration scheme with little cost for analog front-end is developed. Moreover, a partial non-overlapping clock scheme associated with a highspeed reference buffer and fast comparators is proposed to maximize the residue settling time. The implemented ADC is measured under different input frequencies with a sampling rate of 250 MS/s and it consumes 300 mW from a 1.8 V supply. For 30 MHz input, the measured SFDR and SNDR of the ADC is 94.7 dB and 68.5 dB, which can remain over 84.3 dB and 65.4 dB for up to 400 MHz. The measured DNL and INL after calibration are optimized to 0.15 LSB and 1.00 LSB, respectively, while the Walden FOM at Nyquist frequency is 0.57 pJ/step.

    @article{lirolem25371,
              volume = {63},
              number = {9},
               month = {September},
              author = {Xuqiang Zheng and Zhijun Wang and Fule Li and Feng Zhao and Shigang Yue and Chun Zhang and Zhihua Wang},
               title = {A 14-bit 250 MS/s IF sampling pipelined ADC in 180 nm CMOS process},
           publisher = {IEEE},
                year = {2016},
             journal = {IEEE Transactions on Circuits and Systems I: Regular Papers},
               pages = {1381--1392},
            keywords = {ARRAY(0x7f78592c2940)},
                 url = {http://eprints.lincoln.ac.uk/25371/},
            abstract = {This paper presents a 14-bit 250 MS/s ADC fabricated
    in a 180 nm CMOS process, which aims at optimizing its
    linearity, operating speed, and power efficiency. The implemented
    ADC employs an improved SHA with parasitic optimized bootstrapped
    switches to achieve high sampling linearity over a wide
    input frequency range. It also explores a dedicated foreground
    calibration to correct the capacitor mismatches and the gain
    error of residue amplifier, where a novel configuration scheme
    with little cost for analog front-end is developed. Moreover, a
    partial non-overlapping clock scheme associated with a highspeed
    reference buffer and fast comparators is proposed to
    maximize the residue settling time. The implemented ADC is
    measured under different input frequencies with a sampling rate
    of 250 MS/s and it consumes 300 mW from a 1.8 V supply. For 30
    MHz input, the measured SFDR and SNDR of the ADC is 94.7
    dB and 68.5 dB, which can remain over 84.3 dB and 65.4 dB for
    up to 400 MHz. The measured DNL and INL after calibration
    are optimized to 0.15 LSB and 1.00 LSB, respectively, while the
    Walden FOM at Nyquist frequency is 0.57 pJ/step.}
    }

2015

  • A. Abdolmaleki, N. Lau, L. P. Reis, and G. Neumann, “Regularized covariance estimation for weighted maximum likelihood policy search methods,” in Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on, 2015, pp. 154-159.
    [BibTeX] [Abstract] [EPrints]

    Many episode-based (or direct) policy search algorithms, maintain a multivariate Gaussian distribution as search distribution over the parameter space of some objective function. One class of algorithms, such as episodic REPS, PoWER or PI2 uses, a weighted maximum likelihood estimate (WMLE) to update the mean and covariance matrix of this distribution in each iteration. However, due to high dimensionality of covariance matrices and limited number of samples, the WMLE is an unreliable estimator. The use of WMLE leads to over-fitted covariance estimates, and, hence the variance/entropy of the search distribution decreases too quickly, which may cause premature convergence. In order to alleviate this problem, the estimated covariance matrix can be regularized in different ways, for example by using a convex combination of the diagonal covariance estimate and the sample covariance estimate. In this paper, we propose a new covariance matrix regularization technique for policy search methods that uses the convex combination of the sample covariance matrix and the old covariance matrix used in last iteration. The combination weighting is determined by specifying the desired entropy of the new search distribution. With this mechanism, the entropy of the search distribution can be gradually decreased without damage from the maximum likelihood estimate.

    @inproceedings{lirolem25748,
              volume = {2015-D},
               month = {November},
              author = {A. Abdolmaleki and N. Lau and L. P. Reis and G. Neumann},
           booktitle = {Humanoid Robots (Humanoids), 2015 IEEE-RAS 15th International Conference on},
               title = {Regularized covariance estimation for weighted maximum likelihood policy search methods},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {154--159},
                year = {2015},
            keywords = {ARRAY(0x7f78593dcf98)},
                 url = {http://eprints.lincoln.ac.uk/25748/},
            abstract = {Many episode-based (or direct) policy search algorithms, maintain a multivariate Gaussian distribution as search distribution over the parameter space of some objective function. One class of algorithms, such as episodic REPS, PoWER or PI2 uses, a weighted maximum likelihood estimate (WMLE) to update the mean and covariance matrix of this distribution in each iteration. However, due to high dimensionality of covariance matrices and limited number of samples, the WMLE is an unreliable estimator. The use of WMLE leads to over-fitted covariance estimates, and, hence the variance/entropy of the search distribution decreases too quickly, which may cause premature convergence. In order to alleviate this problem, the estimated covariance matrix can be regularized in different ways, for example by using a convex combination of the diagonal covariance estimate and the sample covariance estimate. In this paper, we propose a new covariance matrix regularization technique for policy search methods that uses the convex combination of the sample covariance matrix and the old covariance matrix used in last iteration. The combination weighting is determined by specifying the desired entropy of the new search distribution. With this mechanism, the entropy of the search distribution can be gradually decreased without damage from the maximum likelihood estimate.}
    }
  • A. Abdolmaleki, N. Lau, L. P. Reis, J. Peters, and G. Neumann, “Contextual policy search for generalizing a parameterized biped walking controller,” in IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC), 2015, pp. 17-22.
    [BibTeX] [Abstract] [EPrints]

    We investigate learning of flexible Robot locomotion controller, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed and the direction of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method. With such a contextual policy search algorithm, we can generalize the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear one over the contexts. In order to validate our method, we perform a simulation experiment using a simulated NAO humanoid robot. The robot now learns a policy to choose the controller parameters for a continuous set of walking speeds and directions.

    @inproceedings{lirolem25698,
           booktitle = {IEEE International Conference on  Autonomous Robot Systems and Competitions (ICARSC)},
               month = {April},
               title = {Contextual policy search for generalizing a parameterized biped walking controller},
              author = {A. Abdolmaleki and N. Lau and L. P. Reis and J. Peters and G. Neumann},
           publisher = {IEEE},
                year = {2015},
               pages = {17--22},
            keywords = {ARRAY(0x7f7859443930)},
                 url = {http://eprints.lincoln.ac.uk/25698/},
            abstract = {We investigate learning of flexible Robot locomotion controller, i.e., the controllers should be applicable for multiple contexts, for example different walking speeds, various slopes of the terrain or other physical properties of the robot. In our experiments, contexts are desired walking linear speed and the direction of the gait. Current approaches for learning control parameters of biped locomotion controllers are typically only applicable for a single context. They can be used for a particular context, for example to learn a gait with highest speed, lowest energy consumption or a combination of both. The question of our research is, how can we obtain a flexible walking controller that controls the robot (near) optimally for many different contexts? We achieve the desired flexibility of the controller by applying the recently developed contextual relative entropy policy search(REPS) method. With such a contextual policy search algorithm, we can generalize the robot walking controller for different contexts, where a context is described by a real valued vector. In this paper we also extend the contextual REPS algorithm to learn a non-linear policy instead of a linear one over the contexts. In order to validate our method, we perform a simulation experiment using a simulated NAO humanoid robot. The robot now learns a policy to choose the controller parameters for a continuous set of walking speeds and directions.}
    }
  • S. Albrecht, A. M. S. da Barreto, D. Braziunas, D. Buckeridge, and H. Cuayahuitl, “Reports of the AAAI 2014 Conference Workshops,” AI Magazine, vol. 36, iss. 1, pp. 87-98, 2015.
    [BibTeX] [Abstract] [EPrints]

    The AAAI-14 Workshop program was held Sunday and Monday, July 27?28, 2012, at the Québec City Convention Centre in Québec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive Technologies and Smart Environments; Cognitive Computing for Augmented Human Intelligence; Computer Poker and Imperfect Information; Discovery Informatics; Incentives and Trust in Electronic Communities; Intelligent Cinematography and Editing; Machine Learning for Interactive Systems: Bridging the Gap between Perception, Action and Communication; Modern Artificial Intelligence for Health Analytics; Multiagent Interaction without Prior Coordination; Multidisciplinary Workshop on Advances in Preference Handling; Semantic Cities — Beyond Open Data to Models, Standards and Reasoning; Sequential Decision Making with Big Data; Statistical Relational AI; and The World Wide Web and Public Health Intelligence. This article presents short summaries of those events.

    @article{lirolem22215,
              volume = {36},
              number = {1},
               month = {January},
              author = {Stefano Albrecht and Andr{\'e} da Motta Salles Barreto and Darius Braziunas and David Buckeridge and Heriberto Cuayahuitl},
               title = {Reports of the AAAI 2014 Conference Workshops},
           publisher = {Association for the Advancemant of Artificial Intelligence},
                year = {2015},
             journal = {AI Magazine},
               pages = {87--98},
            keywords = {ARRAY(0x7f7859412c28)},
                 url = {http://eprints.lincoln.ac.uk/22215/},
            abstract = {The AAAI-14 Workshop program was held Sunday and Monday, July 27?28, 2012, at the Qu{\'e}bec City Convention Centre in Qu{\'e}bec, Canada. Canada. The AAAI-14 workshop program included fifteen workshops covering a wide range of topics in artificial intelligence. The titles of the workshops were AI and Robotics; Artificial Intelligence Applied to Assistive Technologies and Smart Environments; Cognitive Computing for Augmented Human Intelligence; Computer Poker and Imperfect Information; Discovery Informatics; Incentives and Trust in Electronic Communities; Intelligent Cinematography and Editing; Machine Learning for Interactive Systems: Bridging the Gap between Perception, Action and Communication; Modern Artificial Intelligence for Health Analytics; Multiagent Interaction without Prior Coordination; Multidisciplinary Workshop on Advances in Preference Handling; Semantic Cities {--} Beyond Open Data to Models, Standards and Reasoning; Sequential Decision Making with Big Data; Statistical Relational AI; and The World Wide Web and Public Health Intelligence. This article presents short summaries of those events.}
    }
  • P. Ardin, M. Mangan, A. Wystrach, and B. Webb, “How variation in head pitch could affect image matching algorithms for ant navigation,” Journal of Comparative Physiology A, vol. 201, iss. 6, pp. 585-597, 2015.
    [BibTeX] [Abstract] [EPrints]

    Desert ants are a model system for animal navigation, using visual memory to follow long routes across both sparse and cluttered environments. Most accounts of this behaviour assume retinotopic image matching, e.g. recovering heading direction by finding a minimum in the image difference function as the viewpoint rotates. But most models neglect the potential image distortion that could result from unstable head motion. We report that for ants running across a short section of natural substrate, the head pitch varies substantially: by over 20 degrees with no load; and 60 degrees when carrying a large food item. There is no evidence of head stabilisation. Using a realistic simulation of the ant?s visual world, we demonstrate that this range of head pitch significantly degrades image matching. The effect of pitch variation can be ameliorated by a memory bank of densely sampled along a route so that an image sufficiently similar in pitch and location is available for comparison. However, with large pitch disturbance, inappropriate memories sampled at distant locations are often recalled and navigation along a route can be adversely affected. Ignoring images obtained at extreme pitches, or averaging images over several pitches, does not significantly improve performance.

    @article{lirolem23586,
              volume = {201},
              number = {6},
               month = {June},
              author = {Paul Ardin and Michael Mangan and Antoine Wystrach and Barbara Webb},
               title = {How variation in head pitch could affect image matching algorithms for ant navigation},
           publisher = {Springer Berlin Heidelberg},
                year = {2015},
             journal = {Journal of Comparative Physiology A},
               pages = {585--597},
            keywords = {ARRAY(0x7f78594436a8)},
                 url = {http://eprints.lincoln.ac.uk/23586/},
            abstract = {Desert ants are a model system for animal navigation, using visual memory to follow long routes across both sparse and cluttered environments. Most accounts of this behaviour assume retinotopic image matching, e.g. recovering heading direction by finding a minimum in the image difference function as the viewpoint rotates. But most models neglect the potential image distortion that could result from unstable head motion. We report that for ants running across a short section of natural substrate, the head pitch varies substantially: by over 20 degrees with no load; and 60 degrees when carrying a large food item. There is no evidence of head stabilisation. Using a realistic simulation of the ant?s visual world, we demonstrate that this range of head pitch significantly degrades image matching. The effect of pitch variation can be ameliorated by a memory bank of densely sampled along a route so that an image sufficiently similar in pitch and location is available for comparison. However, with large pitch disturbance, inappropriate memories sampled at distant locations are often recalled and navigation along a route can be adversely affected. Ignoring images obtained at extreme pitches, or averaging images over several pitches, does not significantly improve performance.}
    }
  • F. Arvin, R. Attar, A. E. Turgut, and S. Yue, “Power-law distribution of long-term experimental data in swarm robotics,” in International Conference on Swarm Intelligence, 2015, pp. 551-559.
    [BibTeX] [Abstract] [EPrints]

    Bio-inspired aggregation is one of the most fundamental behaviours that has been studied in swarm robotic for more than two decades. Biology revealed that the environmental characteristics are very important factors in aggregation of social insects and other animals. In this paper, we study the effects of different environmental factors such as size and texture of aggregation cues using real robots. In addition, we propose a mathematical model to predict the behaviour of the aggregation during an experiment.

    @inproceedings{lirolem17627,
           booktitle = {International Conference on Swarm Intelligence},
               month = {June},
               title = {Power-law distribution of long-term experimental data in swarm robotics},
              author = {Farshad Arvin and Rahman Attar and Ali Emre Turgut and Shigang Yue},
           publisher = {Springer},
                year = {2015},
               pages = {551--559},
            keywords = {ARRAY(0x7f78594435d0)},
                 url = {http://eprints.lincoln.ac.uk/17627/},
            abstract = {Bio-inspired aggregation is one of the most fundamental behaviours that has been 
    studied in swarm robotic for more than two decades. Biology revealed that the 
    environmental characteristics are very important factors in aggregation of social insects and 
    other animals. In this paper, we study the effects of different environmental factors such as 
    size and texture of aggregation cues using real robots. In addition, we propose a 
    mathematical model to predict the behaviour of the aggregation during an experiment.}
    }
  • F. Arvin, C. Xiong, and S. Yue, “Colias-\ensuremath\Phi: an autonomous micro robot for artificial pheromone communication,” International Journal of Mechanical Engineering and Robotics Research, vol. 4, iss. 4, pp. 349-353, 2015.
    [BibTeX] [Abstract] [EPrints]

    Ants pheromone communication is an efficient mechanism which took inspiration from nature. It has been used in various artificial intelligence and multi robotics researches. This paper presents the development of an autonomous micro robot to be used in swarm robotic researches especially in pheromone based communication systems. The robot is an extended version of Colias micro robot with capability of decoding and following artificial pheromone trails. We utilize a low-cost experimental setup to implement pheromone-based scenarios using a flat LCD screen and a USB camera. The results of the performed experiments with group of robots demonstrated the feasibility of Colias-\ensuremath\Phi to be used in pheromone based experiments.

    @article{lirolem19405,
              volume = {4},
              number = {4},
               month = {October},
              author = {Farshad Arvin and Caihua Xiong and Shigang Yue},
               title = {Colias-{\ensuremath{\Phi}}: an autonomous micro robot for artificial pheromone communication},
             journal = {International Journal of Mechanical Engineering and Robotics Research},
               pages = {349--353},
                year = {2015},
            keywords = {ARRAY(0x7f78593b55a8)},
                 url = {http://eprints.lincoln.ac.uk/19405/},
            abstract = {Ants pheromone communication is an efficient mechanism which took inspiration from nature. It has been used in various artificial intelligence and multi robotics researches. This paper presents the development of an autonomous micro robot to be used in swarm robotic researches especially in pheromone based communication systems. The robot is an extended version of Colias micro robot with capability of decoding and following artificial pheromone trails. We utilize a low-cost experimental setup to implement pheromone-based scenarios using a flat LCD screen and a USB camera. The results of the performed experiments with group of robots demonstrated the feasibility of Colias-{\ensuremath{\Phi}} to be used in pheromone based experiments.}
    }
  • F. Arvin, T. Krajnik, A. E. Turgut, and S. Yue, “COS-\ensuremath\Phi: artificial pheromone system for robotic swarms research,” IEEE/RSJ International Conference on Intelligent Robots and Systems 2015, 2015.
    [BibTeX] [Abstract] [EPrints]

    Pheromone-based communication is one of the most effective ways of communication widely observed in nature. It is particularly used by social insects such as bees, ants and termites; both for inter-agent and agent-swarm communications. Due to its effectiveness; artificial pheromones have been adopted in multi-robot and swarm robotic systems for more than a decade. Although, pheromone-based communication was implemented by different means like chemical (use of particular chemical compounds) or physical (RFID tags, light, sound) ways, none of them were able to replicate all the aspects of pheromones as seen in nature. In this paper, we propose a novel artificial pheromone system that is reliable, accurate and it uses off-the-shelf components only — LCD screen and low-cost USB camera. The system allows to simulate several pheromones and their interactions and to change parameters of the pheromones (diffusion, evaporation, etc.) on the fly allowing for controllable experiments. We tested the performance of the system using the Colias platform in single-robot and swarm scenarios. To allow the swarm robotics community to use the system for their research, we provide it as a freely available open-source package.

    @article{lirolem17957,
               month = {September},
               title = {COS-{\ensuremath{\Phi}}: artificial pheromone system for robotic swarms research},
              author = {Farshad Arvin and Tomas Krajnik and Ali Emre Turgut and Shigang Yue},
           publisher = {IEEE},
                year = {2015},
                note = {Conference:
    2015 IEEE/RSJ International Conference on  Intelligent Robots and Systems (IROS 2015), 28 September - 2 October 2015, Hamburg, Germany},
             journal = {IEEE/RSJ International Conference on Intelligent Robots and Systems 2015},
            keywords = {ARRAY(0x7f7859417c20)},
                 url = {http://eprints.lincoln.ac.uk/17957/},
            abstract = {Pheromone-based communication is one of the most effective ways of communication widely observed in nature. It is particularly used by social insects such as bees, ants and termites; both for inter-agent and agent-swarm communications. Due to its effectiveness; artificial pheromones have been adopted in multi-robot and swarm robotic systems for more than a decade. Although, pheromone-based communication was implemented by different means like chemical (use of particular chemical compounds) or physical (RFID tags, light, sound) ways, none of them were able to replicate all the aspects of pheromones as seen in nature. In this paper, we propose a novel artificial pheromone system that is reliable, accurate and it uses off-the-shelf components only -- LCD screen and low-cost USB camera. The system allows to simulate several pheromones and their interactions and to change parameters of the pheromones (diffusion, evaporation, etc.) on the fly allowing for controllable experiments. We tested the performance of the system using the Colias platform in single-robot and swarm scenarios. To allow the swarm robotics community to use the system for their research, we provide it as a freely available open-source package.}
    }
  • W. Chen, C. Xiong, and S. Yue, “Mechanical implementation of kinematic synergy for continual grasping generation of anthropomorphic hand,” IEEE/ASME Transactions on Mechatronics, vol. 20, iss. 3, pp. 1249-1263, 2015.
    [BibTeX] [Abstract] [EPrints]

    The synergy-based motion generation of current anthropomorphic hands generally employ the static posture synergy, which is extracted from quantities of joint trajectory, to design the mechanism or control strategy. Under this framework, the temporal weight sequences of each synergy from pregrasp phase to grasp phase are required for reproducing any grasping task. Moreover, the zero-offset posture has to be preset before starting any grasp. Thus, the whole grasp phase appears to be unlike natural human grasp. Up until now, no work in the literature addresses these issues toward simplifying the continual grasp by only inputting the grasp pattern. In this paper, the kinematic synergies observed in angular velocity profile are employed to design the motion generation mechanism. The kinematic synergy extracted from quantities of grasp tasks is implemented by the proposed eigen cam group in tendon space. The completely continual grasp from the fully extending posture only require averagely rotating the two eigen cam groups one cycle. The change of grasp pattern only depends on respecifying transmission ratio pair for the two eigen cam groups. An illustrated hand prototype is developed based on the proposed design principle and the grasping experiments demonstrate the feasibility of the design method. The potential applications include the prosthetic hand that is controlled by the classified pattern from the bio-signal.

    @article{lirolem17879,
              volume = {20},
              number = {3},
               month = {June},
              author = {Wenbin Chen and Caihua Xiong and Shigang Yue},
               title = {Mechanical implementation of kinematic synergy for continual grasping generation of anthropomorphic hand},
           publisher = {IEEE},
                year = {2015},
             journal = {IEEE/ASME Transactions on Mechatronics},
               pages = {1249--1263},
            keywords = {ARRAY(0x7f7859441218)},
                 url = {http://eprints.lincoln.ac.uk/17879/},
            abstract = {The synergy-based motion generation of current anthropomorphic hands generally employ the static posture synergy, which is extracted from quantities of joint trajectory, to design the mechanism or control strategy. Under this framework, the temporal weight sequences of each synergy from pregrasp phase to grasp phase are required for reproducing any grasping task. Moreover, the zero-offset posture has to be preset before starting any grasp. Thus, the whole grasp phase appears to be unlike natural human grasp. Up until now, no work in the literature addresses these issues toward simplifying the continual grasp by only inputting the grasp pattern. In this paper, the kinematic synergies observed in angular velocity profile are employed to design the motion generation mechanism. The kinematic synergy extracted from quantities of grasp tasks is implemented by the proposed eigen cam group in tendon space. The completely continual grasp from the fully extending posture only require averagely rotating the two eigen cam groups one cycle. The change of grasp pattern only depends on respecifying transmission ratio pair for the two eigen cam groups. An illustrated hand prototype is developed based on the proposed design principle and the grasping experiments demonstrate the feasibility of the design method. The potential applications include the prosthetic hand that is controlled by the classified pattern from the bio-signal.}
    }
  • C. Coppola, O. M. Mozos, and N. Bellotto, “Applying a 3D qualitative trajectory calculus to human action recognition using depth cameras,” in IEEE/RSJ IROS Workshop on Assistance and Service Robotics in a Human Environment, 2015.
    [BibTeX] [Abstract] [EPrints]

    The life span of ordinary people is increasing steadily and many developed countries are facing the big challenge of dealing with an ageing population at greater risk of impairments and cognitive disorders, which hinder their quality of life. Monitoring human activities of daily living (ADLs) is important in order to identify potential health problems and apply corrective strategies as soon as possible. Towards this long term goal, the research here presented is a first step to monitor ADLs using 3D sensors in an Ambient Assisted Living (AAL) environment. In particular, the work here presented adopts a new 3D Qualitative Trajectory Calculus (QTC3D) to represent human actions that belong to such activities, designing and implementing a set of computational tools (i.e. Hidden Markov Models) to learn and classify them from standard datasets. Preliminary results show the good performance of our system and its potential application to a large number of scenarios, including mobile robots for AAL.

    @inproceedings{lirolem18477,
           booktitle = {IEEE/RSJ IROS Workshop on Assistance and Service Robotics in a Human Environment},
               month = {October},
               title = {Applying a 3D qualitative trajectory calculus to human action recognition using depth cameras},
              author = {Claudio Coppola and Oscar Martinez Mozos and Nicola Bellotto},
           publisher = {IEEE},
                year = {2015},
                note = {2015 IEEE/RSJ International Conference on Intelligent Robots and Systems},
            keywords = {ARRAY(0x7f7859459bb8)},
                 url = {http://eprints.lincoln.ac.uk/18477/},
            abstract = {The life span of ordinary people is increasing steadily and many developed countries are facing the big challenge of dealing with an ageing population at greater risk of impairments and cognitive disorders, which hinder their quality of life. Monitoring human activities of daily living (ADLs) is important in order to identify potential health problems and apply corrective strategies as soon as possible. Towards this long term goal, the research here presented is a first step to monitor ADLs using 3D sensors in an Ambient Assisted Living (AAL) environment. In particular, the work here presented adopts a new 3D Qualitative Trajectory Calculus (QTC3D) to represent human actions that belong to such activities, designing and implementing a set of computational tools (i.e. Hidden Markov Models) to learn and classify them from standard datasets. Preliminary results show the good performance of our system and its potential application to a large number of scenarios, including mobile robots for AAL.}
    }
  • H. Cuayahuitl, K. Komatani, and G. Skantze, “Introduction for speech and language for interactive robots,” Computer Speech & Language, vol. 34, iss. 1, pp. 83-86, 2015.
    [BibTeX] [Abstract] [EPrints]

    This special issue includes research articles which apply spoken language processing to robots that interact with human users through speech, possibly combined with other modalities. Robots that can listen to human speech, understand it, interact according to the conveyed meaning, and respond represent major research and technological challenges. Their common aim is to equip robots with natural interaction abilities. However, robotics and spoken language processing are areas that are typically studied within their respective communities with limited communication across disciplinary boundaries. The articles in this special issue represent examples that address the need for an increased multidisciplinary exchange of ideas.

    @article{lirolem22214,
              volume = {34},
              number = {1},
               month = {November},
              author = {Heriberto Cuayahuitl and Kazunori Komatani and Gabriel Skantze},
               title = {Introduction for speech and language for interactive robots},
           publisher = {Elsevier for International Speech Communication Association (ISCA)},
                year = {2015},
             journal = {Computer Speech \& Language},
               pages = {83--86},
            keywords = {ARRAY(0x7f78592bc2d8)},
                 url = {http://eprints.lincoln.ac.uk/22214/},
            abstract = {This special issue includes research articles which apply spoken language processing to robots that interact with human users through speech, possibly combined with other modalities. Robots that can listen to human speech, understand it, interact according to the conveyed meaning, and respond represent major research and technological challenges. Their common aim is to equip robots with natural interaction abilities. However, robotics and spoken language processing are areas that are typically studied within their respective communities with limited communication across disciplinary boundaries. The articles in this special issue represent examples that address the need for an increased multidisciplinary exchange of ideas.}
    }
  • H. Cuayahuitl, S. Keizer, and O. Lemon, “Strategic dialogue management via deep reinforcement learning,” in NIPS Workshop on Deep Reinforcement Learning, 2015.
    [BibTeX] [Abstract] [EPrints]

    Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan—where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53\% win rate versus 3 automated players (`bots’), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27\%, versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities.

    @inproceedings{lirolem25994,
           booktitle = {NIPS Workshop on Deep Reinforcement Learning},
              volume = {abs/16},
               title = {Strategic dialogue management via deep reinforcement learning},
              author = {Heriberto Cuayahuitl and Simon Keizer and Oliver Lemon},
           publisher = {arXiv},
                year = {2015},
             journal = {CoRR},
            keywords = {ARRAY(0x7f78593b5c38)},
                 url = {http://eprints.lincoln.ac.uk/25994/},
            abstract = {Artificially intelligent agents equipped with strategic skills that can negotiate during their interactions with other natural or artificial agents are still underdeveloped. This paper describes a successful application of Deep Reinforcement Learning (DRL) for training intelligent agents with strategic conversational skills, in a situated dialogue setting. Previous studies have modelled the behaviour of strategic agents using supervised learning and traditional reinforcement learning techniques, the latter using tabular representations or learning with linear function approximation. In this study, we apply DRL with a high-dimensional state space to the strategic board game of Settlers of Catan---where players can offer resources in exchange for others and they can also reply to offers made by other players. Our experimental results report that the DRL-based learnt policies significantly outperformed several baselines including random, rule-based, and supervised-based behaviours. The DRL-based policy has a 53\% win rate versus 3 automated players (`bots'), whereas a supervised player trained on a dialogue corpus in this setting achieved only 27\%, versus the same 3 bots. This result supports the claim that DRL is a promising framework for training dialogue systems, and strategic agents with negotiation abilities.}
    }
  • C. Dondrup, N. Bellotto, M. Hanheide, K. Eder, and U. Leonards, “A computational model of human-robot spatial interactions based on a qualitative trajectory calculus,” Robotics, vol. 4, iss. 1, pp. 63-102, 2015.
    [BibTeX] [Abstract] [EPrints]

    In this paper we propose a probabilistic sequential model of Human-Robot Spatial Interaction (HRSI) using a well-established Qualitative Trajectory Calculus (QTC) to encode HRSI between a human and a mobile robot in a meaningful, tractable, and systematic manner. Our key contribution is to utilise QTC as a state descriptor and model HRSI as a probabilistic sequence of such states. Apart from the sole direction of movements of human and robot modelled by QTC, attributes of HRSI like proxemics and velocity profiles play vital roles for the modelling and generation of HRSI behaviour. In this paper, we particularly present how the concept of proxemics can be embedded in QTC to facilitate richer models. To facilitate reasoning on HRSI with qualitative representations, we show how we can combine the representational power of QTC with the concept of proxemics in a concise framework, enriching our probabilistic representation by implicitly modelling distances. We show the appropriateness of our sequential model of QTC by encoding different HRSI behaviours observed in two spatial interaction experiments. We classify these encounters, creating a comparative measurement, showing the representational capabilities of the model.

    @article{lirolem16987,
              volume = {4},
              number = {1},
               month = {March},
              author = {Christian Dondrup and Nicola Bellotto and Marc Hanheide and Kerstin Eder and Ute Leonards},
                note = {This article belongs to the Special Issue Representations and Reasoning for Robotics},
               title = {A computational model of human-robot spatial interactions based on a qualitative trajectory calculus},
           publisher = {MDPI},
                year = {2015},
             journal = {Robotics},
               pages = {63--102},
            keywords = {ARRAY(0x7f78593b6100)},
                 url = {http://eprints.lincoln.ac.uk/16987/},
            abstract = {In this paper we propose a probabilistic sequential model of Human-Robot Spatial Interaction (HRSI) using a well-established Qualitative Trajectory Calculus (QTC) to encode HRSI between a human and a mobile robot in a meaningful, tractable, and systematic manner. Our key contribution is to utilise QTC as a state descriptor and model HRSI as a probabilistic sequence of such states. Apart from the sole direction of movements of human and robot modelled by QTC, attributes of HRSI like proxemics and velocity profiles play vital roles for the modelling and generation of HRSI behaviour. In this paper, we particularly present how the concept of proxemics can be embedded in QTC to facilitate richer models. To facilitate reasoning on HRSI with qualitative representations, we show how we can combine the representational power of QTC with the concept of proxemics in a concise framework, enriching our probabilistic representation by implicitly modelling distances. We show the appropriateness of our sequential model of QTC by encoding different HRSI behaviours observed in two spatial interaction experiments. We classify these encounters, creating a comparative measurement, showing the representational capabilities of the model.}
    }
  • C. Dondrup, N. Bellotto, F. Jovan, and M. Hanheide, “Real-time multisensor people tracking for human-robot spatial interaction,” in Workshop on Machine Learning for Social Robotics at ICRA 2015, 2015.
    [BibTeX] [Abstract] [EPrints]

    All currently used mobile robot platforms are able to navigate safely through their environment, avoiding static and dynamic obstacles. However, in human populated environments mere obstacle avoidance is not sufficient to make humans feel comfortable and safe around robots. To this end, a large community is currently producing human-aware navigation approaches to create a more socially acceptable robot behaviour. Amajorbuilding block for all Human-Robot Spatial Interaction is the ability of detecting and tracking humans in the vicinity of the robot. We present a fully integrated people perception framework, designed to run in real-time on a mobile robot. This framework employs detectors based on laser and RGB-D data and a tracking approach able to fuse multiple detectors using different versions of data association and Kalman filtering. The resulting trajectories are transformed into Qualitative Spatial Relations based on a Qualitative Trajectory Calculus, to learn and classify different encounters using a Hidden Markov Model based representation. We present this perception pipeline, which is fully implemented into the Robot Operating System (ROS), in a small proof of concept experiment. All components are readily available for download, and free to use under the MIT license, to researchers in all fields, especially focussing on social interaction learning by providing different kinds of output, i.e. Qualitative Relations and trajectories.

    @inproceedings{lirolem17545,
           booktitle = {Workshop on Machine Learning for Social Robotics at ICRA 2015},
               month = {May},
               title = {Real-time multisensor people tracking for human-robot spatial interaction},
              author = {Christian Dondrup and Nicola Bellotto and Ferdian Jovan and Marc Hanheide},
           publisher = {ICRA / IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f78594437f8)},
                 url = {http://eprints.lincoln.ac.uk/17545/},
            abstract = {All currently used mobile robot platforms are able to navigate safely through their environment, avoiding static and dynamic obstacles. However, in human populated environments mere obstacle avoidance is not sufficient to make humans feel comfortable and safe around robots. To this end, a large community is currently producing human-aware navigation approaches to create a more socially acceptable robot behaviour. Amajorbuilding block for all Human-Robot Spatial Interaction is the ability of detecting and tracking humans in the vicinity of the robot. We present a fully integrated people perception framework, designed to run in real-time on a mobile robot. This framework employs detectors based on laser and RGB-D data and a tracking approach able to fuse multiple detectors using different versions of data association and Kalman filtering. The resulting trajectories are transformed into Qualitative Spatial Relations based on a Qualitative Trajectory Calculus, to learn and classify different encounters using a Hidden Markov Model based representation. We present this perception pipeline, which is fully implemented into the Robot Operating System (ROS), in a small proof of concept experiment. All components are readily available for download, and free to use under the MIT license, to researchers in all fields, especially focussing on social interaction learning by providing different kinds of output, i.e. Qualitative Relations and trajectories.}
    }
  • M. Ewerton, G. Neumann, R. Lioutikov, H. B. Amor, J. Peters, and G. Maeda, “Learning multiple collaborative tasks with a mixture of interaction primitives,” in International Conference on Robotics and Automation (ICRA), 2015, pp. 1535-1542.
    [BibTeX] [Abstract] [EPrints]

    Robots that interact with humans must learn to not only adapt to different human partners but also to new interactions. Such a form of learning can be achieved by demonstrations and imitation. A recently introduced method to learn interactions from demonstrations is the framework of Interaction Primitives. While this framework is limited to represent and generalize a single interaction pattern, in practice, interactions between a human and a robot can consist of many different patterns. To overcome this limitation this paper proposes a Mixture of Interaction Primitives to learn multiple interaction patterns from unlabeled demonstrations. Specifically the proposed method uses Gaussian Mixture Models of Interaction Primitives to model nonlinear correlations between the movements of the different agents. We validate our algorithm with two experiments involving interactive tasks between a human and a lightweight robotic arm. In the first, we compare our proposed method with conventional Interaction Primitives in a toy problem scenario where the robot and the human are not linearly correlated. In the second, we present a proof-of-concept experiment where the robot assists a human in assembling a box.

    @inproceedings{lirolem25762,
              volume = {2015-J},
              number = {June},
               month = {May},
              author = {Marco Ewerton and Gerhard Neumann and Rudolf Lioutikov and Heni Ben Amor and Jan Peters and Guilherme Maeda},
                note = {cited By 2},
           booktitle = {International Conference on Robotics and Automation (ICRA)},
               title = {Learning multiple collaborative tasks with a mixture of interaction primitives},
           publisher = {IEEE},
                year = {2015},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {1535--1542},
            keywords = {ARRAY(0x7f7859443768)},
                 url = {http://eprints.lincoln.ac.uk/25762/},
            abstract = {Robots that interact with humans must learn to
    not only adapt to different human partners but also to new
    interactions. Such a form of learning can be achieved by
    demonstrations and imitation. A recently introduced method
    to learn interactions from demonstrations is the framework
    of Interaction Primitives. While this framework is limited
    to represent and generalize a single interaction pattern, in
    practice, interactions between a human and a robot can consist
    of many different patterns. To overcome this limitation this
    paper proposes a Mixture of Interaction Primitives to learn
    multiple interaction patterns from unlabeled demonstrations.
    Specifically the proposed method uses Gaussian Mixture Models
    of Interaction Primitives to model nonlinear correlations
    between the movements of the different agents. We validate
    our algorithm with two experiments involving interactive tasks
    between a human and a lightweight robotic arm. In the first,
    we compare our proposed method with conventional Interaction
    Primitives in a toy problem scenario where the robot and the
    human are not linearly correlated. In the second, we present a
    proof-of-concept experiment where the robot assists a human
    in assembling a box.}
    }
  • J. P. Fentanes, B. Lacerda, T. Krajnik, N. Hawes, and M. Hanheide, “Now or later? Predicting and maximising success of navigation actions from long-term experience,” in 2015 IEEE International Conference on Robotics and Automation (ICRA 2015), 2015.
    [BibTeX] [Abstract] [EPrints]

    In planning for deliberation or navigation in real-world robotic systems, one of the big challenges is to cope with change. It lies in the nature of planning that it has to make assumptions about the future state of the world, and the robot?s chances of successively accomplishing actions in this future. Hence, a robot?s plan can only be as good as its predictions about the world. In this paper, we present a novel approach to specifically represent changes that stem from periodic events in the environment (e.g. a door being opened or closed), which impact on the success probability of planned actions. We show that our approach to model the probability of action success as a set of superimposed periodic processes allows the robot to predict action outcomes in a long-term data obtained in two real-life offices better than a static model. We furthermore discuss and showcase how this knowledge gathered can be successfully employed in a probabilistic planning framework to devise better navigation plans. The key contributions of this paper are (i) the formation of the spectral model of action outcomes from non-uniform sampling, the (ii) analysis of its predictive power using two long-term datasets, and (iii) the application of the predicted outcomes in an MDP-based planning framework.

    @inproceedings{lirolem17745,
           booktitle = {2015 IEEE International Conference on Robotics and Automation (ICRA 2015)},
               month = {May},
               title = {Now or later? Predicting and maximising success of navigation actions from long-term experience},
              author = {Jaime Pulido Fentanes and Bruno Lacerda and Tomas Krajnik and Nick Hawes and Marc Hanheide},
           publisher = {IEEE/RAS},
                year = {2015},
            keywords = {ARRAY(0x7f78594438e8)},
                 url = {http://eprints.lincoln.ac.uk/17745/},
            abstract = {In planning for deliberation or navigation in real-world robotic systems, one of the big challenges is to cope with change. It lies in the nature of planning that it has to make assumptions about the future state of the world, and the robot?s chances of successively accomplishing actions in this future.
    Hence, a robot?s plan can only be as good as its predictions about the world. In this paper, we present a novel approach to specifically represent changes that stem from periodic events in the environment (e.g. a door being opened or closed), which impact on the success probability of planned actions. We show that our approach to model the probability of action success as a set of superimposed periodic processes allows the robot to predict action outcomes in a long-term data obtained in two real-life offices better than a static model. We furthermore discuss and showcase how this knowledge gathered can be successfully employed in a probabilistic planning framework to devise better navigation plans. The key contributions of this paper are (i) the formation of the spectral model of action outcomes from non-uniform sampling, the (ii) analysis of its predictive power using two long-term datasets, and (iii) the application of the predicted outcomes in an MDP-based planning framework.}
    }
  • Q. Fu and S. Yue, “Modelling LGMD2 visual neuron system,” in 2015 IEEE International Workshop on Machine Learning for Signal Processing, 2015.
    [BibTeX] [Abstract] [EPrints]

    Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bioinspired nonlinear functions, are introduced in our model, to achieve LGMD2?s collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.

    @inproceedings{lirolem24940,
           booktitle = {2015 IEEE International Workshop on Machine Learning for Signal Processing},
               month = {September},
               title = {Modelling LGMD2 visual neuron system},
              author = {Qinbing Fu and Shigang Yue},
                year = {2015},
            keywords = {ARRAY(0x7f785945cee8)},
                 url = {http://eprints.lincoln.ac.uk/24940/},
            abstract = {Two Lobula Giant Movement Detectors (LGMDs) have been identified in the lobula region of the locust visual system: LGMD1 and LGMD2. LGMD1 had been successfully used in robot navigation to avoid impending collision. LGMD2 also responds to looming stimuli in depth, and shares most the same properties with LGMD1; however, LGMD2 has its specific collision selective responds when dealing with different visual stimulus. Therefore, in this paper, we propose a novel way to model LGMD2, in order to emulate its predicted bio-functions, moreover, to solve some defects of previous LGMD1 computational models. The mechanism of ON and OFF cells, as well as bioinspired nonlinear functions, are introduced in our model, to achieve LGMD2?s collision selectivity. Our model has been tested by a miniature mobile robot in real time. The results suggested this model has an ideal performance in both software and hardware for collision recognition.}
    }
  • P. Gallina, N. Bellotto, and M. D. Luca, “Progressive co-adaptation in human-machine interaction,” in 12th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2015), 2015.
    [BibTeX] [Abstract] [EPrints]

    In this paper we discuss the concept of co-adaptation between a human operator and a machine interface and we summarize its application with emphasis on two different domains, teleoperation and assistive technology. The analysis of the literature reveals that only in few cases the possibility of a temporal evolution of the co-adaptation parameters has been considered. In particular, it has been overlooked the role of time-related indexes that capture changes in motor and cognitive abilities of the human operator. We argue that for a more effective long-term co-adaptation process, the interface should be able to predict and adjust its parameters according to the evolution of human skills and performance. We thus propose a novel approach termed progressive co-adaptation, whereby human performance is continuously monitored and the system makes inferences about changes in the users’ cognitive and motor skills. We illustrate the features of progressive co-adaptation in two possible applications, robotic telemanipulation and active vision for the visually impaired.

    @inproceedings{lirolem17501,
           booktitle = {12th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2015)},
               month = {July},
               title = {Progressive co-adaptation in human-machine interaction},
              author = {Paolo Gallina and Nicola Bellotto and Massimiliano Di Luca},
                year = {2015},
            keywords = {ARRAY(0x7f7859443588)},
                 url = {http://eprints.lincoln.ac.uk/17501/},
            abstract = {In this paper we discuss the concept of co-adaptation between a human operator and a machine interface and we summarize its application with emphasis on two different domains, teleoperation and assistive technology. The analysis of the literature reveals that only in few cases the possibility of a temporal evolution of the co-adaptation parameters has been considered. In particular, it has been overlooked the role of time-related indexes that capture changes in motor and cognitive abilities of the human operator. We argue that for a more effective long-term co-adaptation process, the interface should be able to predict and adjust its parameters according to the evolution of human skills and performance. We thus propose a novel approach termed progressive co-adaptation, whereby human performance is continuously monitored and the system makes inferences about changes in the users' cognitive and motor skills. We illustrate the features of progressive co-adaptation in two possible applications, robotic telemanipulation and active vision for the visually impaired.}
    }
  • Y. Gao, J. Peng, S. Yue, and Y. Zhao, “On the null space property of lq -minimization for 0\ensuremath<q$łeq$1 in compressed sensing,” Journal of Function Spaces, vol. 2015, p. 579853, 2015.
    [BibTeX] [Abstract] [EPrints]

    The paper discusses the relationship between the null space property (NSP) and the lq-minimization in compressed sensing. Several versions of the null space property, that is, the lq stable NSP, the lq robust NSP, and the lq,p robust NSP for 0\ensuremath<p$łeq$q\ensuremath<1 based on the standard lq NSP, are proposed, and their equivalent forms are derived. Consequently, reconstruction results for the lq-minimization can be derived easily under the NSP condition and its equivalent form. Finally, the lq NSP is extended to the lq-synthesis modeling and the mixed l2/lq-minimization, which deals with the dictionary-based sparse signals and the block sparse signals, respectively. \copyright 2015 Yi Gao et al

    @article{lirolem17374,
              volume = {2015},
              author = {Yi Gao and Jigen Peng and Shigang Yue and Yuan Zhao},
                note = {This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
    Journal Title History
    Journal of Function Spaces 2014?Current
    Journal of Function Spaces and Applications 2003?2013 (Title Changed)  (ISSN 2090-8997, eISSN 0972-6802)},
               title = {On the null space property of lq -minimization for 0{\ensuremath{<}}q{$\leq$}1 in compressed sensing},
           publisher = {Hindawi Publishing Corporation},
             journal = {Journal of Function Spaces},
               pages = {579853},
                year = {2015},
            keywords = {ARRAY(0x7f78593b5e90)},
                 url = {http://eprints.lincoln.ac.uk/17374/},
            abstract = {The paper discusses the relationship between the null space property (NSP) and the lq-minimization in compressed sensing. Several versions of the null space property, that is, the lq stable NSP, the lq robust NSP, and the lq,p robust NSP for 0{\ensuremath{<}}p{$\leq$}q{\ensuremath{<}}1 based on the standard lq NSP, are proposed, and their equivalent forms are derived. Consequently, reconstruction results for the lq-minimization can be derived easily under the NSP condition and its equivalent form. Finally, the lq NSP is extended to the lq-synthesis modeling and the mixed l2/lq-minimization, which deals with the dictionary-based sparse signals and the block sparse signals, respectively. {\copyright} 2015 Yi Gao et al}
    }
  • Y. Gao, W. Wang, and S. Yue, “On the rate of convergence by generalized Baskakov operators,” Advances in Mathematical Physics, vol. 2015, p. 564854, 2015.
    [BibTeX] [Abstract] [EPrints]

    We firstly construct generalized Baskakov operators V n, \ensuremath\alpha, q (f; x) and their truncated sum B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x). Secondly, we study the pointwise convergence and the uniform convergence of the operators V n, \ensuremath\alpha, q (f; x), respectively, and estimate that the rate of convergence by the operators V n, \ensuremath\alpha, q (f; x) is 1 / n q / 2. Finally, we study the convergence by the truncated operators B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x) and state that the finite truncated sum B n, \ensuremath\alpha, q (f; \ensuremath\gamma n, x) can replace the operators V n, \ensuremath\alpha, q (f; x) in the computational point of view provided that l i m n $\rightarrow$ ? n \ensuremath\gamma n = ?. \copyright 2015 Yi Gao et al.

    @article{lirolem17367,
              volume = {2015},
               month = {May},
              author = {Yi Gao and W. Wang and Shigang Yue},
                note = {This is an open access article distributed under the Creative Commons Attribution License, which
    permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.},
               title = {On the rate of convergence by generalized Baskakov operators},
           publisher = {Hindawi Publishing Corporation},
                year = {2015},
             journal = {Advances in Mathematical Physics},
               pages = {564854},
            keywords = {ARRAY(0x7f78594439a8)},
                 url = {http://eprints.lincoln.ac.uk/17367/},
            abstract = {We firstly construct generalized Baskakov operators V n, {\ensuremath{\alpha}}, q (f; x) and their truncated sum B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x). Secondly, we study the pointwise convergence and the uniform convergence of the operators V n, {\ensuremath{\alpha}}, q (f; x), respectively, and estimate that the rate of convergence by the operators V n, {\ensuremath{\alpha}}, q (f; x) is 1 / n q / 2. Finally, we study the convergence by the truncated operators B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x) and state that the finite truncated sum B n, {\ensuremath{\alpha}}, q (f; {\ensuremath{\gamma}} n, x) can replace the operators V n, {\ensuremath{\alpha}}, q (f; x) in the computational point of view provided that l i m n {$\rightarrow$} ? n {\ensuremath{\gamma}} n = ?. {\copyright} 2015 Yi Gao et al.}
    }
  • P. Graham and M. Mangan, “Insect navigation: do ants live in the now?,” Journal of Experimental Biology, vol. 218, iss. 6, pp. 819-823, 2015.
    [BibTeX] [Abstract] [EPrints]

    Visual navigation is a critical behaviour formanyanimals, and it has been particularly well studied in ants. Decades of ant navigation research have uncovered many ways in which efficient navigation can be implemented in small brains. For example, ants show us how visual information can drive navigation via procedural rather than map-like instructions. Two recent behavioural observations highlight interesting adaptive ways in which ants implement visual guidance. Firstly, it has been shownthat the systematic nest searches of ants can be biased by recent experience of familiar scenes. Secondly, ants have been observed to show temporary periods of confusion when asked to repeat a route segment, even if that route segment is very familiar. Taken together, these results indicate that the navigational decisions of ants take into account their recent experiences as well as the currently perceived environment.

    @article{lirolem23585,
              volume = {218},
              number = {6},
               month = {March},
              author = {Paul Graham and Michael Mangan},
               title = {Insect navigation: do ants live in the now?},
           publisher = {Company of Biologists},
                year = {2015},
             journal = {Journal of Experimental Biology},
               pages = {819--823},
            keywords = {ARRAY(0x7f78592e0900)},
                 url = {http://eprints.lincoln.ac.uk/23585/},
            abstract = {Visual navigation is a critical behaviour formanyanimals, and it has been
    particularly well studied in ants. Decades of ant navigation research have
    uncovered many ways in which efficient navigation can be implemented
    in small brains. For example, ants show us how visual information can
    drive navigation via procedural rather than map-like instructions. Two
    recent behavioural observations highlight interesting adaptive ways in
    which ants implement visual guidance. Firstly, it has been shownthat the
    systematic nest searches of ants can be biased by recent experience of
    familiar scenes. Secondly, ants have been observed to show temporary
    periods of confusion when asked to repeat a route segment, even if that
    route segment is very familiar. Taken together, these results indicate that
    the navigational decisions of ants take into account their recent
    experiences as well as the currently perceived environment.}
    }
  • E. Gyebi, F. Arvin, M. Hanheide, S. Yue, and G. Cielniak, “Colias: towards an affordable mobile robot for education in developing countries,” in Developing Countries Forum at ICRA 2015, 2015.
    [BibTeX] [Abstract] [EPrints]

    Educational robotics can play a key role in addressing some of the important challenges faced by higher education in developing countries. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in these parts of the world is a lack of robot platforms which would be affordable for the local educational institutions. In this paper, we present our inexpensive mobile robot platform Colias and assess its potential for education in developing countries. To this end, we describe hardware and software components of the robot, assess its suitability for education and discuss the missing features which will need to be developed to turn Colias into a fully featured educational platform. The presented robot is one of the key components of our current efforts in popularising educational robotics at African universities.

    @inproceedings{lirolem17558,
           booktitle = {Developing Countries Forum at ICRA 2015},
               month = {May},
               title = {Colias: towards an affordable mobile robot for education in developing countries},
              author = {Ernest Gyebi and Farshad Arvin and Marc Hanheide and Shigang Yue and Grzegorz Cielniak},
                year = {2015},
            keywords = {ARRAY(0x7f78594437c8)},
                 url = {http://eprints.lincoln.ac.uk/17558/},
            abstract = {Educational robotics can play a key role in addressing some of the important challenges faced by higher education
    in developing countries. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in these parts of the world is a lack of robot platforms which would be affordable for the local educational institutions. In this paper, we present our inexpensive mobile robot platform Colias and assess its potential for education in developing countries. To this end, we describe hardware and software components of the robot, assess its suitability for education and discuss the missing features which will need to be developed to turn Colias into a fully featured educational platform. The presented robot is one of the key components of our current efforts in popularising
    educational robotics at African universities.}
    }
  • E. Gyebi, M. Hanheide, and G. Cielniak, “Educational robotics for teaching computer science in Africa – pilot study,” in WONDER 2015, First International Workshop on Educational Robotics, 2015.
    [BibTeX] [Abstract] [EPrints]

    Educational robotics can play a key role in addressing some of the challenges faced by higher education institutions in Africa. A remaining and open question is related to effectiveness of activities involving educational robots for teaching but also for improving learner’s experience. This paper addresses that question by evaluating a short pilot study which introduced students at the Department of Computer Science, University of Ghana to robot programming. The initial positive results from the study indicate a potential for such activities to enhance teaching experience and practice at African institutions. The proposed integrated set-up including robotic hardware, software and educational tasks was effective and will form a solid base for a future, full scale integration of robotic activities into the undergraduate curricula at this particular institution. This evaluation should be valuable to other educators integrating educational robots into undergraduate curricula in developing countries and elsewhere.

    @inproceedings{lirolem19407,
           booktitle = {WONDER 2015, First International Workshop on Educational Robotics},
               month = {October},
               title = {Educational robotics for teaching computer science in Africa - pilot study},
              author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
                year = {2015},
            keywords = {ARRAY(0x7f78593b53f8)},
                 url = {http://eprints.lincoln.ac.uk/19407/},
            abstract = {Educational robotics can play a key role in addressing some of the challenges faced by higher education institutions in Africa. A remaining and open question is related to effectiveness of activities involving educational robots for teaching but also for improving learner's experience. This paper addresses that question by evaluating a short pilot study which introduced students at the Department of Computer Science, University of Ghana to robot programming. The initial positive results from the study indicate a potential for such activities to enhance teaching experience and practice at African institutions. The proposed integrated set-up including robotic hardware, software and educational tasks was effective and will form a solid base for a future, full scale integration of robotic activities into the undergraduate curricula at this particular institution. This evaluation should be valuable to other educators integrating educational robots into undergraduate curricula in developing countries and elsewhere.}
    }
  • E. Gyebi, M. Hanheide, and G. Cielniak, “Affordable mobile robotic platforms for teaching computer science at African universities,” in 6th International Conference on Robotics in Education, 2015.
    [BibTeX] [Abstract] [EPrints]

    Educational robotics can play a key role in addressing some of the challenges faced by higher education in Africa. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in this part of the world is lack of robots that would be affordable by African institutions. In this paper, we present a survey and analysis of currently available affordable mobile robots and their suitability for teaching computer science at African universities. To this end, we propose a set of assessment criteria and review a number of platforms costing an order of magnitude less than the existing popular educational robots. Our analysis identifies suitable candidates offering contrasting features and benefits. We also discuss potential issues and promising directions which can be considered by both educators in Africa but also designers and manufacturers of future robot platforms.

    @inproceedings{lirolem17557,
           booktitle = {6th International Conference on Robotics in Education},
               month = {May},
               title = {Affordable mobile robotic platforms for teaching computer science at African universities},
              author = {Ernest Gyebi and Marc Hanheide and Grzegorz Cielniak},
                year = {2015},
            keywords = {ARRAY(0x7f78594438a0)},
                 url = {http://eprints.lincoln.ac.uk/17557/},
            abstract = {Educational robotics can play a key role in addressing some of the challenges faced by higher education in Africa. One of the major obstacles preventing a wider adoption of initiatives involving educational robotics in this part of the world is lack of robots that would be affordable by African institutions. In this paper, we present a survey and analysis of currently available affordable mobile robots and their suitability for teaching computer science at African universities. To this end, we propose a set of assessment criteria and review a number of platforms costing an order of magnitude less than the existing popular educational robots. Our analysis identifies suitable candidates offering contrasting features and benefits. We also discuss potential issues and promising directions which can be considered by both educators in Africa but also designers and manufacturers of future robot platforms.}
    }
  • D. Hebesberger, T. Körtner, J. Pripfl, C. Gisinger, and M. Hanheide, “What do staff in eldercare want a robot for? An assessment of potential tasks and user requirements for a long-term deployment,” in IROS Workshop on "Bridging user needs to deployed applications of service robots", Hamburg, 2015.
    [BibTeX] [Abstract] [EPrints]

    Robotic aids could help to overcome the gap between rising numbers of older adults and at the same time declining numbers of care staff. Assessments of end-user requirements, especially focusing on staff in eldercare facilities are still sparse. Contributing to this field of research this study presents end-user requirements and task analysis gained from a methodological combination of interviews and focus group discussions. The findings suggest different tasks robots in eldercare could engage in such as ?fetch and carry? tasks, specific entertainment and information tasks, support in physical and occupational therapy, and in security. Furthermore this paper presents an iterative approach that closes the loop between requirements-assessments and subsequent implementations that follow the found requirements.

    @inproceedings{lirolem18860,
           booktitle = {IROS Workshop on "Bridging user needs to deployed applications of service robots"},
               month = {September},
               title = {What do staff in eldercare want a robot for? An assessment of potential tasks and user requirements for a long-term deployment},
              author = {Denise Hebesberger and Tobias K{\"o}rtner and J{\"u}rgen Pripfl and Christoph Gisinger and Marc Hanheide},
             address = {Hamburg},
                year = {2015},
                note = {The Robot-Era Project has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement num. 288899 
    FP7 - ICT - Challenge 5: ICT for Health, Ageing Well, Inclusion and Governance},
            keywords = {ARRAY(0x7f78593a1118)},
                 url = {http://eprints.lincoln.ac.uk/18860/},
            abstract = {Robotic aids could help to overcome the gap between rising numbers of older adults and at the same time declining numbers of care staff. Assessments of end-user requirements, especially focusing on staff in eldercare facilities are still sparse. Contributing to this field of research this study presents end-user requirements and task analysis gained from a methodological combination of interviews and focus group discussions. The findings suggest different tasks robots in eldercare could engage in such as ?fetch and carry? tasks, specific entertainment and information tasks, support in physical and occupational therapy, and in security. Furthermore this paper presents an iterative approach that closes the loop between requirements-assessments and subsequent implementations that follow the found requirements.}
    }
  • H. V. Hoof, J. Peters, and G. Neumann, “Learning of non-parametric control policies with high-dimensional state features,” Journal of Machine Learning Research: Workshop and Conference Proceedings, vol. 38, pp. 995-1003, 2015.
    [BibTeX] [Abstract] [EPrints]

    Learning complex control policies from highdimensional sensory input is a challenge for reinforcement learning algorithms. Kernel methods that approximate values functions or transition models can address this problem. Yet, many current approaches rely on instable greedy maximization. In this paper, we develop a policy search algorithm that integrates robust policy updates and kernel embeddings. Our method can learn nonparametric control policies for infinite horizon continuous MDPs with high-dimensional sensory representations. We show that our method outperforms related approaches, and that our algorithm can learn an underpowered swing-up task task directly from highdimensional image data.

    @article{lirolem25757,
              volume = {38},
               month = {May},
              author = {Herke Van Hoof and Jan Peters and Gerhard Neumann},
                note = {Proceedings of the 18th International Conference
    on Artificial Intelligence and Statistics (AISTATS), 9-12 May
    2015, San Diego, CA,},
           booktitle = {18th International Conference on Artificial Intelligence and Statistics (AISTATS)},
               title = {Learning of non-parametric control policies with high-dimensional state features},
           publisher = {MIT Press},
                year = {2015},
             journal = {Journal of Machine Learning Research: Workshop and Conference Proceedings},
               pages = {995--1003},
            keywords = {ARRAY(0x7f7859443978)},
                 url = {http://eprints.lincoln.ac.uk/25757/},
            abstract = {Learning complex control policies from highdimensional sensory input is a challenge for
    reinforcement learning algorithms. Kernel methods that approximate values functions
    or transition models can address this problem. Yet, many current approaches rely on
    instable greedy maximization. In this paper, we develop a policy search algorithm that
    integrates robust policy updates and kernel embeddings. Our method can learn nonparametric
    control policies for infinite horizon continuous MDPs with high-dimensional
    sensory representations. We show that our method outperforms related approaches, and
    that our algorithm can learn an underpowered swing-up task task directly from highdimensional
    image data.}
    }
  • V. H. Hoof, T. Hermans, G. Neumann, and J. Peters, “Learning robot in-hand manipulation with tactile features,” in International Conference on Humanoid Robots (HUMANOIDS), 2015, pp. 121-127.
    [BibTeX] [Abstract] [EPrints]

    Dexterous manipulation enables repositioning of objects and tools within a robot?s hand. When applying dexterous manipulation to unknown objects, exact object models are not available. Instead of relying on models, compliance and tactile feedback can be exploited to adapt to unknown objects. However, compliant hands and tactile sensors add complexity and are themselves difficult to model. Hence, we propose acquiring in-hand manipulation skills through reinforcement learning, which does not require analytic dynamics or kinematics models. In this paper, we show that this approach successfully acquires a tactile manipulation skill using a passively compliant hand. Additionally, we show that the learned tactile skill generalizes to novel objects.

    @inproceedings{lirolem25750,
              volume = {2015-D},
               month = {November},
              author = {H. Van Hoof and T. Hermans and G. Neumann and J. Peters},
           booktitle = {International Conference on Humanoid Robots (HUMANOIDS)},
               title = {Learning robot in-hand manipulation with tactile features},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {121--127},
                year = {2015},
            keywords = {ARRAY(0x7f78592e2db0)},
                 url = {http://eprints.lincoln.ac.uk/25750/},
            abstract = {Dexterous manipulation enables repositioning of
    objects and tools within a robot?s hand. When applying dexterous
    manipulation to unknown objects, exact object models
    are not available. Instead of relying on models, compliance and
    tactile feedback can be exploited to adapt to unknown objects.
    However, compliant hands and tactile sensors add complexity
    and are themselves difficult to model. Hence, we propose acquiring
    in-hand manipulation skills through reinforcement learning,
    which does not require analytic dynamics or kinematics models.
    In this paper, we show that this approach successfully acquires
    a tactile manipulation skill using a passively compliant hand.
    Additionally, we show that the learned tactile skill generalizes
    to novel objects.}
    }
  • J. Kennedy, P. Baxter, and T. Belpaeme, “The robot who tried too hard: social behaviour of a robot tutor can negatively affect child learning,” in Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction – HRI ’15, 2015, pp. 67-74.
    [BibTeX] [Abstract] [EPrints]

    Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context.

    @inproceedings{lirolem24856,
           booktitle = {Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction - HRI '15},
               month = {March},
               title = {The robot who tried too hard: social behaviour of a robot tutor can negatively affect child learning},
              author = {James Kennedy and Paul Baxter and Tony Belpaeme},
           publisher = {ACM},
                year = {2015},
               pages = {67--74},
            keywords = {ARRAY(0x7f78593b6178)},
                 url = {http://eprints.lincoln.ac.uk/24856/},
            abstract = {Social robots are finding increasing application in the domain of education, particularly for children, to support and augment learning opportunities. With an implicit assumption that social and adaptive behaviour is desirable, it is therefore of interest to determine precisely how these aspects of behaviour may be exploited in robots to support children in their learning. In this paper, we explore this issue by evaluating the effect of a social robot tutoring strategy with children learning about prime numbers. It is shown that the tutoring strategy itself leads to improvement, but that the presence of a robot employing this strategy amplifies this effect, resulting in significant learning. However, it was also found that children interacting with a robot using social and adaptive behaviours in addition to the teaching strategy did not learn a significant amount. These results indicate that while the presence of a physical robot leads to improved learning, caution is required when applying social behaviour to a robot in a tutoring context.}
    }
  • J. Kennedy, P. Baxter, and T. Belpaeme, “Comparing robot embodiments in a guided discovery learning interaction with children,” International Journal of Social Robotics, vol. 7, iss. 2, pp. 293-308, 2015.
    [BibTeX] [Abstract] [EPrints]

    The application of social robots to the domain of education is becoming more prevalent. However, there re- main a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the em- bodiment of a robot influences the interaction. In this paper, we seek to explore children?s behaviour towards a robot tutor for children in a novel guided discovery learning interac- tion. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant dif- ferences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits

    @article{lirolem23075,
              volume = {7},
              number = {2},
               month = {April},
              author = {James Kennedy and Paul Baxter and Tony Belpaeme},
               title = {Comparing robot embodiments in a guided discovery learning interaction with children},
           publisher = {Springer verlag},
                year = {2015},
             journal = {International Journal of Social Robotics},
               pages = {293--308},
            keywords = {ARRAY(0x7f78593edca8)},
                 url = {http://eprints.lincoln.ac.uk/23075/},
            abstract = {The application of social robots to the domain of education is becoming more prevalent. However, there re- main a wide range of open issues, such as the effectiveness of robots as tutors on student learning outcomes, the role of social behaviour in teaching interactions, and how the em- bodiment of a robot influences the interaction. In this paper, we seek to explore children?s behaviour towards a robot tutor for children in a novel guided discovery learning interac- tion. Since the necessity of real robots (as opposed to virtual agents) in education has not been definitively established in the literature, the effect of robot embodiment is assessed. The results demonstrate that children overcome strong incorrect biases in the material to be learned, but with no significant dif- ferences between embodiment conditions. However, the data do suggest that the use of real robots carries an advantage in terms of social presence that could provide educational benefits}
    }
  • A. Kodzhabashev and M. Mangan, “Route following without scanning,” in Biomimetic and Biohybrid Systems: 4th International Conference, Living Machines 2015,, 2015, pp. 199-210.
    [BibTeX] [Abstract] [EPrints]

    Desert ants are expert navigators, foraging over large distances using visually guided routes. Recent models of route following can reproduce aspects of route guidance, yet the underlying motor patterns do not reflect those of foraging ants. Specifically, these models select the direction of movement by rotating to find the most familiar view. Yet scanning patterns are only occasionally observed in ants. We propose a novel route following strategy inspired by klinokinesis. By using familiarity of the view to modulate the magnitude of alternating left and right turns, and the size of forward steps, this strategy is able to continually correct the heading of a simulated ant to maintain its course along a route. Route following by klinokinesis and visual compass are evaluated against real ant routes in a simulation study and on a mobile robot in the real ant habitat. We report that in unfamiliar surroundings the proposed method can also generate ant-like scanning behaviours.

    @inproceedings{lirolem24845,
           booktitle = {Biomimetic and Biohybrid Systems: 4th International Conference, Living Machines 2015,},
               month = {July},
               title = {Route following without scanning},
              author = {Aleksandar Kodzhabashev and Michael Mangan},
           publisher = {Springer International Publishing},
                year = {2015},
               pages = {199--210},
            keywords = {ARRAY(0x7f785941e1a8)},
                 url = {http://eprints.lincoln.ac.uk/24845/},
            abstract = {Desert ants are expert navigators, foraging over large distances using visually guided routes. Recent models of route following can reproduce aspects of route guidance, yet the underlying motor patterns do not reflect those of foraging ants. Specifically, these models select the direction of movement by rotating to find the most familiar view. Yet scanning patterns are only occasionally observed in ants. We propose a novel route following strategy inspired by klinokinesis. By using familiarity of the view to modulate the magnitude of alternating left and right turns, and the size of forward steps, this strategy is able to continually correct the heading of a simulated ant to maintain its course along a route. Route following by klinokinesis and visual compass are evaluated against real ant routes in a simulation study and on a mobile robot in the real ant habitat. We report that in unfamiliar surroundings the proposed method can also generate ant-like scanning behaviours.}
    }
  • T. Krajnik, P. deCristoforis, M. Nitsche, K. Kusumam, and T. Duckett, “Image features and seasons revisited,” in European Conference on Mobile Robots 2015 (ECMR 15), 2015.
    [BibTeX] [Abstract] [EPrints]

    We present an evaluation of standard image features in the context of long-term visual teach-and-repeat mobile robot navigation, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that in the given long-term scenario, the viewpoint, scale and rotation invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes. We evaluate the image feature extractors on three datasets collected by mobile robots in two different outdoor environments over the course of one year. Based on this analysis, we propose a novel feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the GRIEF feature descriptor outperforms the other ones while being computationally more efficient.

    @inproceedings{lirolem17954,
           booktitle = {European Conference on Mobile Robots 2015 (ECMR 15)},
               month = {September},
               title = {Image features and seasons revisited},
              author = {Tomas Krajnik and Pablo deCristoforis and Matias Nitsche and Keerthy Kusumam and Tom Duckett},
           publisher = {IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f78593b55d8)},
                 url = {http://eprints.lincoln.ac.uk/17954/},
            abstract = {We present an evaluation of standard image features in the context of long-term visual teach-and-repeat mobile robot navigation, where the environment exhibits significant changes in appearance caused by seasonal weather variations and daily illumination changes. We argue that in the given long-term scenario, the viewpoint, scale and rotation invariance of the standard feature extractors is less important than their robustness to the mid- and long-term environment appearance changes. Therefore, we focus our evaluation on the robustness of image registration to variable lighting and naturally-occurring seasonal changes.  We evaluate the image feature extractors on three datasets collected by mobile robots in two different outdoor environments over the course of one year. Based on this analysis, we propose a novel feature descriptor based on a combination of evolutionary algorithms and Binary Robust Independent Elementary Features, which we call GRIEF (Generated BRIEF). In terms of robustness to seasonal changes, the GRIEF feature descriptor outperforms the other ones while being computationally more efficient.}
    }
  • T. Krajnik, J. Santos, and T. Duckett, “Life-long spatio-temporal exploration of dynamic environments,” in European Conference on Mobile Robots 2015 (ECMR 15), 2015.
    [BibTeX] [Abstract] [EPrints]

    We propose a new idea for life-long mobile robot spatio-temporal exploration of dynamic environments. Our method assumes that the world is subject to perpetual change, which adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process. To create and maintain a spatio-temporal model of a dynamic environment, the robot has to determine not only where, but also when to perform observations. We address the problem by application of information-theoretic exploration to world representations that model the uncertainty of environment states as probabilistic functions of time. We compare the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months and show that combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of constantly changing environments.

    @inproceedings{lirolem17955,
           booktitle = {European Conference on Mobile Robots 2015 (ECMR 15)},
               month = {September},
               title = {Life-long spatio-temporal exploration of dynamic environments},
              author = {Tomas Krajnik and Joao Santos and Tom Duckett},
           publisher = {IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f78593b5d88)},
                 url = {http://eprints.lincoln.ac.uk/17955/},
            abstract = {We propose a new idea for life-long mobile robot spatio-temporal exploration of dynamic environments.  Our method assumes that the world is subject to perpetual change, which adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process. To create and maintain a spatio-temporal model of a dynamic environment, the robot has to determine not only where, but also when to perform observations.  We address the problem by application of information-theoretic exploration to world representations that model the uncertainty of environment states as probabilistic functions of time.
    
    We compare the performance of different exploration strategies and temporal models on real-world data gathered over the course of several months and show that combination of dynamic environment representations with information-gain exploration principles allows to create and maintain up-to-date models of constantly changing environments.}
    }
  • T. Krajnik, F. Arvin, A. E. Turgut, S. Yue, and T. Duckett, “COS\ensuremath\Phi: Vision-based artificial pheromone system for robotic swarms,” in IEEE International Conference on Robotics and Automation (ICRA 2015), 2015.
    [BibTeX] [Abstract] [EPrints]

    We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment’s structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot’s environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states’ uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations. To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.

    @inproceedings{lirolem17952,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA 2015)},
               month = {May},
               title = {COS{\ensuremath{\Phi}}: Vision-based artificial pheromone system for robotic swarms},
              author = {Tomas Krajnik and Farshad Arvin and Ali Emre Turgut and Shigang Yue and Tom Duckett},
           publisher = {IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f7859443780)},
                 url = {http://eprints.lincoln.ac.uk/17952/},
            abstract = {We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time.  Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date.
    Thus, the crucial question is not only where, but also when to observe the explored environment. 
    We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations. 
    
    To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot.  Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.}
    }
  • T. Krajnik, M. Kulich, L. Mudrova, R. Ambrus, and T. Duckett, “Where’s Waldo at time t? Using spatio-temporal models for mobile robot search,” in IEEE International Conference on Robotics and Automation (ICRA), 2015, pp. 2140-2146.
    [BibTeX] [Abstract] [EPrints]

    We present a novel approach to mobile robot search for non-stationary objects in partially known environments. We formulate the search as a path planning problem in an environment where the probability of object occurrences at particular locations is a function of time. We propose to explicitly model the dynamics of the object occurrences by their frequency spectra. Using this spectral model, our path planning algorithm can construct plans that reflect the likelihoods of object locations at the time the search is performed. Three datasets collected over several months containing person and object occurrences in residential and office environments were chosen to evaluate the approach. Several types of spatio-temporal models were created for each of these datasets and the efficiency of the search method was assessed by measuring the time it took to locate a particular object. The results indicate that modeling the dynamics of object occurrences reduces the search time by 25\% to 65\% compared to maps that neglect these dynamics.

    @inproceedings{lirolem17949,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
               month = {May},
               title = {Where's Waldo at time t? Using spatio-temporal models for mobile robot search},
              author = {Tomas Krajnik and Miroslav Kulich and Lenka Mudrova and Rares Ambrus and Tom Duckett},
           publisher = {Institute of Electrical and Electronics Engineers},
                year = {2015},
               pages = {2140--2146},
            keywords = {ARRAY(0x7f7859443858)},
                 url = {http://eprints.lincoln.ac.uk/17949/},
            abstract = {We present a novel approach to mobile robot search for non-stationary objects in partially known environments. We formulate the search as a path planning problem in an environment where the probability of object occurrences at particular locations is a function of time. We propose to explicitly model the dynamics of the object occurrences by their frequency spectra. Using this spectral model, our path planning algorithm can construct plans that reflect the likelihoods of object locations at the time the search is performed. Three datasets collected over several months containing person and object occurrences in residential and office environments were chosen to evaluate the approach. Several types of spatio-temporal models were created for each of these datasets and the efficiency of the search method was assessed by measuring the time it took to locate a particular object. The results indicate that modeling the dynamics of object occurrences reduces the search time by 25\% to 65\% compared to maps that neglect these dynamics.}
    }
  • T. Krajnik, J. P. Fentanes, J. Santos, K. Kusumam, and T. Duckett, “FreMEn: frequency map enhancement for long-term mobile robot autonomy in changing environments,” in ICRA 2015 Workshop on Visual Place Recognition in Changing Environments, 2015.
    [BibTeX] [Abstract] [EPrints]

    We present a method for introducing representation of dynamics into environment models that were originally tailored to represent static scenes. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by probabilistic functions of time. These are composed of combinations of harmonic functions, which are obtained by means of frequency analysis. The use of frequency analysis allows to integrate long-term observations into memory-efficient spatio-temporal models that reflect the mid- to long-term environment dynamics. These frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments. In a series of experiments performed over periods of days to years, we demonstrate that the proposed approach improves localization, path planning and exploration.

    @inproceedings{lirolem17953,
           booktitle = {ICRA 2015 Workshop on Visual Place Recognition in Changing Environments},
               month = {May},
               title = {FreMEn: frequency map enhancement for long-term mobile robot autonomy in changing environments},
              author = {Tomas Krajnik and Jaime Pulido Fentanes and Joao Santos and Keerthy Kusumam and Tom Duckett},
           publisher = {IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f7859443888)},
                 url = {http://eprints.lincoln.ac.uk/17953/},
            abstract = {We present a method for introducing representation of dynamics into environment models that were originally tailored to represent static scenes. Rather than using a fixed probability value, the method models the uncertainty of the elementary environment states by probabilistic functions of time. These are composed of combinations of harmonic functions, which are obtained by means of frequency analysis. The use of frequency analysis allows to integrate long-term observations into memory-efficient spatio-temporal models that reflect the mid- to long-term environment dynamics. These frequency-enhanced spatio-temporal models allow to predict the future environment states, which improves the efficiency of mobile robot operation in changing environments.   In a series of experiments performed over periods of days to years, we demonstrate that the proposed approach improves localization, path planning and exploration.}
    }
  • O. Kroemer, C. Daniel, G. Neumann, H. V. Hoof, and J. Peters, “Towards learning hierarchical skills for multi-phase manipulation tasks,” in International Conference on Robotics and Automation (ICRA), 2015, pp. 1503-1510.
    [BibTeX] [Abstract] [EPrints]

    Most manipulation tasks can be decomposed into a sequence of phases, where the robot?s actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

    @inproceedings{lirolem25759,
              volume = {2015-J},
              number = {June},
               month = {June},
              author = {Oliver Kroemer and Christian Daniel and Gerhard Neumann and Herke Van Hoof and Jan Peters},
           booktitle = {International Conference on Robotics and Automation (ICRA)},
               title = {Towards learning hierarchical skills for multi-phase manipulation tasks},
           publisher = {IEEE},
                year = {2015},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {1503--1510},
            keywords = {ARRAY(0x7f78594436d8)},
                 url = {http://eprints.lincoln.ac.uk/25759/},
            abstract = {Most manipulation tasks can be decomposed into
    a sequence of phases, where the robot?s actions have different
    effects in each phase. The robot can perform actions to
    transition between phases and, thus, alter the effects of its
    actions, e.g. grasp an object in order to then lift it. The robot
    can thus reach a phase that affords the desired manipulation.
    In this paper, we present an approach for exploiting the
    phase structure of tasks in order to learn manipulation skills
    more efficiently. Starting with human demonstrations, the robot
    learns a probabilistic model of the phases and the phase
    transitions. The robot then employs model-based reinforcement
    learning to create a library of motor primitives for transitioning
    between phases. The learned motor primitives generalize to new
    situations and tasks. Given this library, the robot uses a value
    function approach to learn a high-level policy for sequencing
    the motor primitives. The proposed method was successfully
    evaluated on a real robot performing a bimanual grasping task.}
    }
  • O. Kroemer, C. Daniel, G. Neumann, V. H. Hoof, and J. Peters, “Towards learning hierarchical skills for multi-phase manipulation tasks,” in IEEE International Conference on Robotics and Automation (ICRA), 2015, 2015, pp. 1503-1510.
    [BibTeX] [Abstract] [EPrints]

    Most manipulation tasks can be decomposed into a sequence of phases, where the robot’s actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.

    @inproceedings{lirolem25696,
              volume = {2015-J},
              number = {June},
               month = {May},
              author = {O. Kroemer and C. Daniel and G. Neumann and H. Van Hoof and J. Peters},
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA), 2015},
               title = {Towards learning hierarchical skills for multi-phase manipulation tasks},
           publisher = {IEEE},
                year = {2015},
               pages = {1503--1510},
            keywords = {ARRAY(0x7f7859443810)},
                 url = {http://eprints.lincoln.ac.uk/25696/},
            abstract = {Most manipulation tasks can be decomposed into a sequence of phases, where the robot's actions have different effects in each phase. The robot can perform actions to transition between phases and, thus, alter the effects of its actions, e.g. grasp an object in order to then lift it. The robot can thus reach a phase that affords the desired manipulation. In this paper, we present an approach for exploiting the phase structure of tasks in order to learn manipulation skills more efficiently. Starting with human demonstrations, the robot learns a probabilistic model of the phases and the phase transitions. The robot then employs model-based reinforcement learning to create a library of motor primitives for transitioning between phases. The learned motor primitives generalize to new situations and tasks. Given this library, the robot uses a value function approach to learn a high-level policy for sequencing the motor primitives. The proposed method was successfully evaluated on a real robot performing a bimanual grasping task.}
    }
  • H. Li, J. Peng, and S. Yue, “The sparsity of underdetermined linear system via lp minimization for 0 \ensuremath< p \ensuremath< 1,” Mathematical Problems in Engineering, vol. 2015, 2015.
    [BibTeX] [Abstract] [EPrints]

    The sparsity problems have attracted a great deal of attention in recent years, which aim to find the sparsest solution of a representation or an equation. In the paper, we mainly study the sparsity of underdetermined linear system via lp minimization for 0\ensuremath<p\ensuremath<1. We show, for a given underdetermined linear system of equations pm$\times$np = p, that although it is not certain that the problem (pp) (i.e., minlx\ensuremath|\ensuremath|X\ensuremath|\ensuremath|plp subject to pp = b, where 0\ensuremath<p\ensuremath<1 ) generates sparser solutions as the value of p decreases and especially the problem (plp) generates sparser solutions than the problem (p1) (i.e., minlx\ensuremath|\ensuremath|X\ensuremath|\ensuremath|1 subject to AX = b ), there exists a sparse constant \ensuremath\gamma(A, p) \ensuremath> 0 such that the following conclusions hold when p \ensuremath< \ensuremath\gamma(A, b): (1) the problem (pp) generates sparser solution as the value of p decreases; (2) the sparsest optimal solution to the problem (pp) is unique under the sense of absolute value permutation; (3) let X1 and X2 be the sparsest optimal solution to the problems (pp1) and (pp2) , respectively, and let X1 not be the absolute value permutation of X2. Then there exist t1,t2 \ensuremath\epsilon [p1,p2] such that X1 is the sparsest optimal solution to the problem (pt) (?t \ensuremath\epsilon [p1, t1]) and X2 is the sparsest optimal solution to the problem (pt) (?t \ensuremath\epsilon (t2, p2]).

    @article{lirolem17577,
              volume = {2015},
               month = {June},
              author = {Haiyang Li and Jigen Peng and Shigang Yue},
                note = {Article ID 584712, 6 pages},
               title = {The sparsity of underdetermined linear system via lp minimization for 0 {\ensuremath{<}} p {\ensuremath{<}} 1},
           publisher = {Hindawi Publishing Corporation},
             journal = {Mathematical Problems in Engineering},
                year = {2015},
            keywords = {ARRAY(0x7f7859443660)},
                 url = {http://eprints.lincoln.ac.uk/17577/},
            abstract = {The sparsity problems have attracted a great deal of attention in recent years, which aim to find the sparsest solution of a representation or an equation. In the paper, we mainly study the sparsity of underdetermined linear system via lp minimization for 0{\ensuremath{<}}p{\ensuremath{<}}1. We show, for a given underdetermined linear system of equations pm{$\times$}np = p, that although it is not certain that the problem (pp) (i.e., minlx{\ensuremath{|}}{\ensuremath{|}}X{\ensuremath{|}}{\ensuremath{|}}plp subject to pp = b, where  0{\ensuremath{<}}p{\ensuremath{<}}1 ) generates sparser solutions as the value of p decreases and especially the problem (plp) generates sparser solutions than the problem (p1) (i.e., minlx{\ensuremath{|}}{\ensuremath{|}}X{\ensuremath{|}}{\ensuremath{|}}1 subject to AX = b ), there exists a sparse constant {\ensuremath{\gamma}}(A, p) {\ensuremath{>}} 0 such that the following conclusions hold when p {\ensuremath{<}} {\ensuremath{\gamma}}(A, b): (1) the problem (pp) generates sparser solution as the value of p decreases; (2) the sparsest optimal solution to the problem (pp) is unique under the sense of absolute value permutation; (3) let X1 and X2 be the sparsest optimal solution to the problems (pp1) and (pp2) , respectively, and let  X1 not be the absolute value permutation of  X2. Then there exist t1,t2 {\ensuremath{\epsilon}} [p1,p2]  such that X1 is the sparsest optimal solution to the problem (pt) (?t {\ensuremath{\epsilon}} [p1, t1])  and X2 is the sparsest optimal solution to the problem (pt) (?t {\ensuremath{\epsilon}} (t2, p2]).}
    }
  • P. Lightbody, C. Dondrup, and M. Hanheide, “Make me a sandwich! Intrinsic human identification from their course of action,” in Towards a Framework for Joint Action, 2015.
    [BibTeX] [Abstract] [EPrints]

    In order to allow humans and robots to work closely together and as a team, we need to equip robots not only with a general understanding of joint action, but also with an understanding of the idiosyncratic differences in the ways humans perform certain tasks. This will allow robots to be better colleagues, by anticipating an individual’s actions, and acting accordingly. In this paper, we present a way of encoding a human’s course of action as a probabilistic sequence of qualitative states, and show that such a model can be employed to identify individual humans from their respective course of action, even when accomplishing the very same goal state. We conclude from our findings that there are significant variations in the ways humans accomplish the very same task, and that our representation could in future work inform robot (task) planning in collaborative settings.

    @inproceedings{lirolem19696,
           booktitle = {Towards a Framework for Joint Action},
               month = {October},
               title = {Make me a sandwich! Intrinsic human identification from their course of action},
              author = {Peter Lightbody and Christian Dondrup and Marc Hanheide},
                year = {2015},
            keywords = {ARRAY(0x7f7859407660)},
                 url = {http://eprints.lincoln.ac.uk/19696/},
            abstract = {In order to allow humans and robots to work closely together and as a team, we need to equip robots not only with a general understanding of joint action, but also with an understanding of the idiosyncratic differences in the ways humans perform certain tasks. This will allow robots to be better colleagues, by anticipating an individual's actions, and acting accordingly. In this paper, we present a way of encoding a human's course of action as a probabilistic sequence of qualitative states, and show that such a model can be employed to identify individual humans from their respective course of action, even when accomplishing the very same goal state. We conclude from our findings that there are significant variations in the ways humans accomplish the very same task, and that our representation could in future work inform robot (task) planning in collaborative settings.}
    }
  • R. Lioutikov, G. Neumann, G. Maeda, and J. Peters, “Probabilistic segmentation applied to an assembly task,” in 15th IEEE-RAS International Conference on Humanoid Robots, 2015, pp. 533-540.
    [BibTeX] [Abstract] [EPrints]

    Movement primitives are a well established approach for encoding and executing robot movements. While the primitives themselves have been extensively researched, the concept of movement primitive libraries has not received as much attention. Libraries of movement primitives represent the skill set of an agent and can be queried and sequenced in order to solve specific tasks. The goal of this work is to segment unlabeled demonstrations into an optimal set of skills. Our novel approach segments the demonstrations while learning a probabilistic representation of movement primitives. The method differs from current approaches by taking advantage of the often neglected, mutual dependencies between the segments contained in the demonstrations and the primitives to be encoded. Therefore, improving the combined quality of both segmentation and skill learning. Furthermore, our method allows incorporating domain specific insights using heuristics, which are subsequently evaluated and assessed through probabilistic inference methods. We demonstrate our method on a real robot application, where the robot segments demonstrations of a chair assembly task into a skill library. The library is subsequently used to assemble the chair in an order not present in the demonstrations.

    @inproceedings{lirolem25751,
              volume = {2015-D},
               month = {November},
              author = {R. Lioutikov and G. Neumann and G. Maeda and J. Peters},
           booktitle = {15th IEEE-RAS International Conference on Humanoid Robots},
               title = {Probabilistic segmentation applied to an assembly task},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {533--540},
                year = {2015},
            keywords = {ARRAY(0x7f78592da498)},
                 url = {http://eprints.lincoln.ac.uk/25751/},
            abstract = {Movement primitives are a well established approach
    for encoding and executing robot movements. While
    the primitives themselves have been extensively researched, the
    concept of movement primitive libraries has not received as
    much attention. Libraries of movement primitives represent
    the skill set of an agent and can be queried and sequenced in
    order to solve specific tasks. The goal of this work is to segment
    unlabeled demonstrations into an optimal set of skills. Our
    novel approach segments the demonstrations while learning
    a probabilistic representation of movement primitives. The
    method differs from current approaches by taking advantage of
    the often neglected, mutual dependencies between the segments
    contained in the demonstrations and the primitives to be encoded.
    Therefore, improving the combined quality of both segmentation
    and skill learning. Furthermore, our method allows
    incorporating domain specific insights using heuristics, which
    are subsequently evaluated and assessed through probabilistic
    inference methods. We demonstrate our method on a real robot
    application, where the robot segments demonstrations of a chair
    assembly task into a skill library. The library is subsequently
    used to assemble the chair in an order not present in the
    demonstrations.}
    }
  • N. Mavridis, N. Bellotto, K. Iliopoulos, and N. V. de Weghe, “QTC3D: extending the qualitative trajectory calculus to three dimensions,” Information Sciences, vol. 322, pp. 20-30, 2015.
    [BibTeX] [Abstract] [EPrints]

    Spatial interactions between agents (humans, animals, or machines) carry information of high value to human or electronic observers. However, not all the information contained in a pair of continuous trajectories is important and thus the need for qualitative descriptions of interaction trajectories arises. The Qualitative Trajectory Calculus (QTC) (Van de Weghe, 2004) is a promising development towards this goal. Numerous variants of QTC have been proposed in the past and QTC has been applied towards analyzing various interaction domains. However, an inherent limitation of those QTC variations that deal with lateral movements is that they are limited to two-dimensional motion; therefore, complex three-dimensional interactions, such as those occurring between flying planes or birds, cannot be captured. Towards that purpose, in this paper QTC3D is presented: a novel qualitative trajectory calculus that can deal with full three-dimensional interactions. QTC3D is based on transformations of the Frenet-Serret frames accompanying the trajectories of the moving objects. Apart from the theoretical exposition, including definition and properties, as well as computational aspects, we also present an application of QTC3D towards modeling bird flight. Thus, the power of QTC is now extended to the full dimensionality of physical space, enabling succinct yet rich representations of spatial interactions between agents.

    @article{lirolem17596,
              volume = {322},
               month = {November},
              author = {Nikolaos Mavridis and Nicola Bellotto and Konstantinos Iliopoulos and Nico Van de Weghe},
               title = {QTC3D: extending the qualitative trajectory calculus to three dimensions},
           publisher = {Elsevier},
             journal = {Information Sciences},
               pages = {20--30},
                year = {2015},
            keywords = {ARRAY(0x7f78593dcdd0)},
                 url = {http://eprints.lincoln.ac.uk/17596/},
            abstract = {Spatial interactions between agents (humans, animals, or machines) carry information of high value to human or electronic observers. However, not all the information contained in a pair of continuous trajectories is important and thus the need for qualitative descriptions of interaction trajectories arises. The Qualitative Trajectory Calculus (QTC) (Van de Weghe, 2004) is a promising development towards this goal. Numerous variants of QTC have been proposed in the past and QTC has been applied towards analyzing various interaction domains. However, an inherent limitation of those QTC variations that deal with lateral movements is that they are limited to two-dimensional motion; therefore, complex three-dimensional interactions, such as those occurring between flying planes or birds, cannot be captured. Towards that purpose, in this paper QTC3D is presented: a novel qualitative trajectory calculus that can deal with full three-dimensional interactions. QTC3D is based on transformations of the Frenet-Serret frames accompanying the trajectories of the moving objects. Apart from the theoretical exposition, including definition and properties, as well as computational aspects, we also present an application of QTC3D towards modeling bird flight. Thus, the power of QTC is now extended to the full dimensionality of physical space, enabling succinct yet rich representations of spatial interactions between agents.}
    }
  • M. Milford, H. Kim, M. Mangan, S. Leutenegger, T. Stone, B. Webb, and A. Davison, “Place recognition with event-based cameras and a neural implementation of SeqSLAM,” OALib Journal, 2015.
    [BibTeX] [Abstract] [EPrints]

    Event-based cameras (Figure 1) offer much potential to the fields of robotics and computer vision, in part due to their large dynamic range and extremely high ?frame rates?. These attributes make them, at least in theory, particularly suitable for enabling tasks like navigation and mapping on high speed robotic platforms under challenging lighting conditions, a task which has been particularly challenging for traditional algorithms and camera sensors. Before these tasks become feasible however, progress must be made towards adapting and innovating current RGB-camera-based algorithms to work with eventbased cameras. In this paper we present ongoing research investigating two distinct approaches to incorporating event-based cameras for robotic navigation: 1. The investigation of suitable place recognition / loop closure techniques, and 2. The development of efficient neural implementations of place recognition techniques that enable the possibility of place recognition using event-based cameras at very high frame rates using neuromorphic computing hardware. Figure 1: The first commercial event camera: (a) DVS128; (b) a stream of events (upward and downward spikes: positive and negative events); (c) image-like visualisation of accumulated events within a time interval (white and black: positive and negative events). From (H. Kim, 2014)].

    @article{lirolem23587,
               title = {Place recognition with event-based cameras and a neural implementation of SeqSLAM},
              author = {Michael Milford and Hanme Kim and Michael Mangan and Stefan Leutenegger and Tom Stone and Barbara Webb and Andrew Davison},
           publisher = {Open Access Library},
                year = {2015},
                note = {arXiv preprint arXiv:1505.04548},
             journal = {OALib Journal},
            keywords = {ARRAY(0x7f78593b6190)},
                 url = {http://eprints.lincoln.ac.uk/23587/},
            abstract = {Event-based cameras (Figure 1) offer much potential to the fields of robotics and computer
    vision, in part due to their large dynamic range and extremely high ?frame rates?. These
    attributes make them, at least in theory, particularly suitable for enabling tasks like
    navigation and mapping on high speed robotic platforms under challenging lighting
    conditions, a task which has been particularly challenging for traditional algorithms and
    camera sensors. Before these tasks become feasible however, progress must be made
    towards adapting and innovating current RGB-camera-based algorithms to work with eventbased
    cameras. In this paper we present ongoing research investigating two distinct
    approaches to incorporating event-based cameras for robotic navigation:
    1. The investigation of suitable place recognition / loop closure techniques, and
    2. The development of efficient neural implementations of place recognition
    techniques that enable the possibility of place recognition using event-based
    cameras at very high frame rates using neuromorphic computing hardware.
    Figure 1: The first commercial event camera: (a) DVS128; (b) a stream of events (upward and
    downward spikes: positive and negative events); (c) image-like visualisation of accumulated
    events within a time interval (white and black: positive and negative events). From (H. Kim,
    2014)].}
    }
  • M. Nitsche, T. Krajnik, P. Cizek, M. Mejail, and T. Duckett, “WhyCon: an efficient, marker-based localization system,” in IROS Workshop on Aerial Open-source Robotics, 2015.
    [BibTeX] [Abstract] [EPrints]

    We present an open-source marker-based localization system intended as a low-cost easy-to-deploy solution for aerial and swarm robotics. The main advantage of the presented method is its high computational efficiency, which allows its deployment on small robots with limited computational resources. Even on low-end computers, the core component of the system can detect and estimate 3D positions of hundreds of black and white markers at the maximum frame-rate of standard cameras. The method is robust to changing lighting conditions and achieves accuracy in the order of millimeters to centimeters. Due to its reliability, simplicity of use and availability as an open-source ROS module (http://purl.org/robotics/whycon), the system is now used in a number of aerial robotics projects where fast and precise relative localization is required.

    @inproceedings{lirolem18877,
           booktitle = {IROS Workshop on Aerial Open-source Robotics},
               month = {September},
               title = {WhyCon: an efficient, marker-based localization system},
              author = {Matias Nitsche and Tomas Krajnik and Petr Cizek and Marta Mejail and Tom Duckett},
                year = {2015},
            keywords = {ARRAY(0x7f78592cca30)},
                 url = {http://eprints.lincoln.ac.uk/18877/},
            abstract = {We present an open-source marker-based localization system intended as a low-cost easy-to-deploy solution for aerial and swarm robotics. The main advantage of the presented method is its high computational efficiency, which allows its deployment on small robots with limited computational resources. Even on low-end computers, the core component of the system can detect and estimate 3D positions of hundreds of black and white markers at the maximum frame-rate of standard cameras. The method is robust to changing lighting conditions and achieves accuracy in the order of millimeters to centimeters. Due to its reliability, simplicity of use and availability as an open-source ROS module (http://purl.org/robotics/whycon), the system is now used in a number of aerial robotics projects where fast and precise relative localization is required.}
    }
  • A. Paraschos, E. Rueckert, J. Peters, and G. Neumann, “Model-free Probabilistic Movement Primitives for physical interaction,” in IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2015, pp. 2860-2866.
    [BibTeX] [Abstract] [EPrints]

    Physical interaction in robotics is a complex problem that requires not only accurate reproduction of the kinematic trajectories but also of the forces and torques exhibited during the movement. We base our approach on Movement Primitives (MP), as MPs provide a framework for modelling complex movements and introduce useful operations on the movements, such as generalization to novel situations, time scaling, and others. Usually, MPs are trained with imitation learning, where an expert demonstrates the trajectories. However, MPs used in physical interaction either require additional learning approaches, e.g., reinforcement learning, or are based on handcrafted solutions. Our goal is to learn and generate movements for physical interaction that are learned with imitation learning, from a small set of demonstrated trajectories. The Probabilistic Movement Primitives (ProMPs) framework is a recent MP approach that introduces beneficial properties, such as combination and blending of MPs, and represents the correlations present in the movement. The ProMPs provides a variable stiffness controller that reproduces the movement but it requires a dynamics model of the system. Learning such a model is not a trivial task, and, therefore, we introduce the model-free ProMPs, that are learning jointly the movement and the necessary actions from a few demonstrations. We derive a variable stiffness controller analytically. We further extent the ProMPs to include force and torque signals, necessary for physical interaction. We evaluate our approach in simulated and real robot tasks.

    @inproceedings{lirolem25752,
              volume = {2015-D},
               month = {September},
              author = {A. Paraschos and E. Rueckert and J. Peters and G. Neumann},
           booktitle = {IEEE/RSJ Conference on Intelligent Robots and Systems (IROS)},
               title = {Model-free Probabilistic Movement Primitives for physical interaction},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
               pages = {2860--2866},
                year = {2015},
            keywords = {ARRAY(0x7f78593dce48)},
                 url = {http://eprints.lincoln.ac.uk/25752/},
            abstract = {Physical interaction in robotics is a complex problem
    that requires not only accurate reproduction of the kinematic
    trajectories but also of the forces and torques exhibited
    during the movement. We base our approach on Movement
    Primitives (MP), as MPs provide a framework for modelling
    complex movements and introduce useful operations on the
    movements, such as generalization to novel situations, time
    scaling, and others. Usually, MPs are trained with imitation
    learning, where an expert demonstrates the trajectories. However,
    MPs used in physical interaction either require additional
    learning approaches, e.g., reinforcement learning, or are based
    on handcrafted solutions. Our goal is to learn and generate
    movements for physical interaction that are learned with imitation
    learning, from a small set of demonstrated trajectories.
    The Probabilistic Movement Primitives (ProMPs) framework
    is a recent MP approach that introduces beneficial properties,
    such as combination and blending of MPs, and represents the
    correlations present in the movement. The ProMPs provides
    a variable stiffness controller that reproduces the movement
    but it requires a dynamics model of the system. Learning such
    a model is not a trivial task, and, therefore, we introduce the
    model-free ProMPs, that are learning jointly the movement and
    the necessary actions from a few demonstrations. We derive
    a variable stiffness controller analytically. We further extent
    the ProMPs to include force and torque signals, necessary for
    physical interaction. We evaluate our approach in simulated
    and real robot tasks.}
    }
  • A. Paraschos, G. Neumann, and J. Peters, “A probabilistic approach to robot trajectory generation,” in International Conference on Humanoid Robots (HUMANOIDS), 2015, pp. 477-483.
    [BibTeX] [Abstract] [EPrints]

    Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations — a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.

    @inproceedings{lirolem25755,
              volume = {2015-F},
              number = {Februa},
               month = {February},
              author = {A. Paraschos and Gerhard Neumann and J. Peters},
           booktitle = {International Conference on Humanoid Robots (HUMANOIDS)},
               title = {A probabilistic approach to robot trajectory generation},
           publisher = {IEEE},
                year = {2015},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {477--483},
            keywords = {ARRAY(0x7f78592e40c0)},
                 url = {http://eprints.lincoln.ac.uk/25755/},
            abstract = {Motor Primitives (MPs) are a promising approach
    for the data-driven acquisition as well as for the modular and
    re-usable generation of movements. However, a modular control
    architecture with MPs is only effective if the MPs support
    co-activation as well as continuously blending the activation
    from one MP to the next. In addition, we need efficient
    mechanisms to adapt a MP to the current situation. Common
    approaches to movement primitives lack such capabilities or
    their implementation is based on heuristics. We present a
    probabilistic movement primitive approach that overcomes the
    limitations of existing approaches. We encode a primitive as a
    probability distribution over trajectories. The representation as
    distribution has several beneficial properties. It allows encoding
    a time-varying variance profile. Most importantly, it allows
    performing new operations {--} a product of distributions for
    the co-activation of MPs conditioning for generalizing the MP
    to different desired targets. We derive a feedback controller
    that reproduces a given trajectory distribution in closed form.
    We compare our approach to the existing state-of-the art and
    present real robot results for learning from demonstration.}
    }
  • J. Peng, S. Yue, and H. Li, “NP/CMP equivalence: a phenomenon hidden among sparsity models l\_\0\ minimization and l\_\p\ minimization for information processing,” IEEE Transactions on Information Theory, vol. 61, iss. 7, pp. 4028-4033, 2015.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we have proved that in every underdetermined linear system Ax = b, there corresponds a constant p*(A, b) \ensuremath> 0 such that every solution to the l p-norm minimization problem also solves the l0-norm minimization problem whenever 0 \ensuremath<; p \ensuremath<; p*(A, b). This phenomenon is named NP/CMP equivalence.

    @article{lirolem17877,
              volume = {61},
              number = {7},
               month = {June},
              author = {Jigen Peng and Shigang Yue and Haiyang Li},
               title = {NP/CMP equivalence: a phenomenon hidden among sparsity models l\_\{0\} minimization and l\_\{p\} minimization for information processing},
           publisher = {IEEE},
                year = {2015},
             journal = {IEEE Transactions on Information Theory},
               pages = {4028--4033},
            keywords = {ARRAY(0x7f7859443618)},
                 url = {http://eprints.lincoln.ac.uk/17877/},
            abstract = {In this paper, we have proved that in every underdetermined linear system Ax = b, there corresponds a constant p*(A, b) {\ensuremath{>}} 0 such that every solution to the l p-norm minimization problem also solves the l0-norm minimization problem whenever 0 {\ensuremath{<}}; p {\ensuremath{<}}; p*(A, b). This phenomenon is named NP/CMP equivalence.}
    }
  • E. Rueckert, J. Mundo, A. Paraschos, J. Peters, and G. Neumann, “Extracting low-dimensional control variables for movement primitives,” in IEEE International Conference on Robotics and Automation 2015, 2015, pp. 1511-1518.
    [BibTeX] [Abstract] [EPrints]

    Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.

    @inproceedings{lirolem25760,
              volume = {2015-J},
              number = {June},
               month = {May},
              author = {E. Rueckert and J. Mundo and A. Paraschos and J. Peters and Gerhard Neumann},
           booktitle = {IEEE International Conference on Robotics and Automation 2015},
               title = {Extracting low-dimensional control variables for movement primitives},
                year = {2015},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {1511--1518},
            keywords = {ARRAY(0x7f78594436f0)},
                 url = {http://eprints.lincoln.ac.uk/25760/},
            abstract = {Movement primitives (MPs) provide a powerful framework for data driven movement generation that has been successfully applied for learning from demonstrations and robot reinforcement learning. In robotics we often want to solve a multitude of different, but related tasks. As the parameters of the primitives are typically high dimensional, a common practice for the generalization of movement primitives to new tasks is to adapt only a small set of control variables, also called meta parameters, of the primitive. Yet, for most MP representations, the encoding of these control variables is pre-coded in the representation and can not be adapted to the considered tasks. In this paper, we want to learn the encoding of task-specific control variables also from data instead of relying on fixed meta-parameter representations. We use hierarchical Bayesian models (HBMs) to estimate a low dimensional latent variable model for probabilistic movement primitives (ProMPs), which is a recent movement primitive representation. We show on two real robot datasets that ProMPs based on HBMs outperform standard ProMPs in terms of generalization and learning from a small amount of data and also allows for an intuitive analysis of the movement. We also extend our HBM by a mixture model, such that we can model different movement types in the same dataset.}
    }
  • V. Sandulescu, S. Andrews, D. Ellis, N. Bellotto, and O. M. Mozos, “Stress detection using wearable physiological sensors,” Lecture Notes in Computer Science, vol. 9107, pp. 526-532, 2015.
    [BibTeX] [Abstract] [EPrints]

    As the population increases in the world, the ratio of health carers is rapidly decreasing. Therefore, there is an urgent need to create new technologies to monitor the physical and mental health of people during their daily life. In particular, negative mental states like depression and anxiety are big problems in modern societies, usually due to stressful situations during everyday activities including work. This paper presents a machine learning approach for stress detection on people using wearable physiological sensors with the ?final aim of improving their quality of life. The presented technique can monitor the state of the subject continuously and classify it into "stressful" or "non-stressful" situations. Our classification results show that this method is a good starting point towards real-time stress detection.

    @article{lirolem17143,
              volume = {9107},
               month = {June},
              author = {Virginia Sandulescu and Sally Andrews and David Ellis and Nicola Bellotto and Oscar Martinez Mozos},
                note = {Series: Lecture Notes in Computer Science
    Artificial Computation in Biology and Medicine: International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2015, Elche, Spain, June 1-5, 2015, Proceedings, Part I},
               title = {Stress detection using wearable physiological sensors},
           publisher = {Springer verlag},
                year = {2015},
             journal = {Lecture Notes in Computer Science},
               pages = {526--532},
            keywords = {ARRAY(0x7f7859443738)},
                 url = {http://eprints.lincoln.ac.uk/17143/},
            abstract = {As the population increases in the world, the ratio of health carers is rapidly decreasing. Therefore, there is an urgent need to create new technologies to monitor the physical and mental health of people during their  daily life. In particular, negative mental states like depression and anxiety are big problems in modern societies, usually due to stressful situations during everyday activities including work. This paper presents a machine learning approach for stress detection on people using wearable physiological sensors with the ?final aim of improving their quality of life. The presented technique can monitor the state of the subject continuously and classify it into "stressful" or "non-stressful" situations. Our classification results show that this method is a good starting point towards real-time stress detection.}
    }
  • J. Santos, T. Krajnik, J. P. Fentanes, and T. Duckett, “Lifelong exploration of dynamic environments,” in IEEE International Conference on Robotics and Automation (ICRA), 2015.
    [BibTeX] [Abstract] [EPrints]

    We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments. In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment’s structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot’s environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states’ uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations. To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.

    @inproceedings{lirolem17951,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
               month = {May},
               title = {Lifelong exploration of dynamic environments},
              author = {Joao Santos and Tomas Krajnik and Jaime Pulido Fentanes and Tom Duckett},
           publisher = {IEEE},
                year = {2015},
            keywords = {ARRAY(0x7f7859443918)},
                 url = {http://eprints.lincoln.ac.uk/17951/},
            abstract = {We propose a novel spatio-temporal mobile-robot exploration method for dynamic, human-populated environments.
    In contrast to other exploration methods that model the environment as being static, our spatio-temporal exploration method creates and maintains a world model that not only represents the environment's structure, but also its dynamics over time. Consideration of the world dynamics adds an extra, temporal dimension to the explored space and makes the exploration task a never-ending data-gathering process to keep the robot's environment model up-to-date. Thus, the crucial question is not only where, but also when to observe the explored environment. We address the problem by application of information-theoretic exploration to world representations that model the environment states' uncertainties as probabilistic functions of time. The predictive ability of the spatio-temporal model allows the exploration method to decide not only where, but also when to make environment observations.
    
    To verify the proposed approach, an evaluation of several exploration strategies and spatio-temporal models was carried out using real-world data gathered over several months. The evaluation indicates that through understanding of the environment dynamics, the proposed spatio-temporal exploration method could predict which locations were going to change at a specific time and use this knowledge to guide the robot. Such an ability is crucial for long-term deployment of mobile robots in human-populated spaces that change over time.}
    }
  • D. Wang, S. Yue, J. Xu, X. Hou, and C. Liu, “A saliency-based cascade method for fast traffic sign detection,” in Intelligent Vehicles Symposium, IV 2015, 2015, pp. 180-185.
    [BibTeX] [Abstract] [EPrints]

    We propose a cascade method for fast and accurate traffic sign detection. The main feature of the method is that mid-level saliency test is used to efficiently and reliably eliminate background windows. Fast feature extraction is adopted in the subsequent stages for rejecting more negatives. Combining with neighbor scales awareness in window search, the proposed method runs at 3\texttt\char1265 fps for high resolution (1360×800) images, 2\texttt\char1267 times as fast as most state-of-the-art methods. Compared with them, the proposed method yields competitive performance on prohibitory signs while sacrifices performance moderately on danger and mandatory signs. \copyright 2015 IEEE.

    @inproceedings{lirolem20151,
              volume = {2015-A},
               month = {July},
              author = {Dongdong Wang and Shigang Yue and Jiawei Xu and Xinwen Hou and Cheng-Lin Liu},
                note = {Conference Code:117127},
           booktitle = {Intelligent Vehicles Symposium, IV 2015},
               title = {A saliency-based cascade method for fast traffic sign detection},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
                year = {2015},
             journal = {IEEE Intelligent Vehicles Symposium, Proceedings},
               pages = {180--185},
            keywords = {ARRAY(0x7f78594435b8)},
                 url = {http://eprints.lincoln.ac.uk/20151/},
            abstract = {We propose a cascade method for fast and accurate traffic sign detection. The main feature of the method is that mid-level saliency test is used to efficiently and reliably eliminate background windows. Fast feature extraction is adopted in the subsequent stages for rejecting more negatives. Combining with neighbor scales awareness in window search, the proposed method runs at 3{\texttt{\char126}}5 fps for high resolution (1360x800) images, 2{\texttt{\char126}}7 times as fast as most state-of-the-art methods. Compared with them, the proposed method yields competitive performance on prohibitory signs while sacrifices performance moderately on danger and mandatory signs. {\copyright} 2015 IEEE.}
    }
  • A. Wystrach, M. Mangan, and B. Webb, “Optimal cue integration in ants,” Proceedings of the Royal Society B: Biological Sciences, vol. 282, iss. 1816, 2015.
    [BibTeX] [Abstract] [EPrints]

    In situations with redundant or competing sensory information, humans have been shown to perform cue integration, weighting different cues according to their certainty in a quantifiably optimal manner. Ants have been shown to merge the directional information available from their path integration (PI) and visual memory, but as yet it is not clear that they do so in a way that reflects the relative certainty of the cues. In this study, we manipulate the variance of the PI home vector by allowing ants (Cataglyphis velox) to run different distances and testing their directional choice when the PI vector direction is put in competition with visual memory. Ants show progressively stronger weighting of their PI direction as PI length increases. The weighting is quantitatively predicted by modelling the expected directional variance of home vectors of different lengths and assuming optimal cue integration. However, a subsequent experiment suggests ants may not actually compute an internal estimate of the PI certainty, but are using the PI home vector length as a proxy.

    @article{lirolem23589,
              volume = {282},
              number = {1816},
               month = {October},
              author = {Antoine Wystrach and Michael Mangan and Barbara Webb},
               title = {Optimal cue integration in ants},
           publisher = {Royal Society},
             journal = {Proceedings of the Royal Society B: Biological Sciences},
                year = {2015},
            keywords = {ARRAY(0x7f78593b33b0)},
                 url = {http://eprints.lincoln.ac.uk/23589/},
            abstract = {In situations with redundant or competing sensory information, humans have been shown to perform cue integration, weighting different cues according to their certainty in a quantifiably optimal manner. Ants have been shown to merge the directional information available from their path integration (PI) and visual memory, but as yet it is not clear that they do so in a way that reflects the relative certainty of the cues. In this study, we manipulate the variance of the PI home vector by allowing ants (Cataglyphis velox) to run different distances and testing their directional choice when the PI vector direction is put in competition with visual memory. Ants show progressively stronger weighting of their PI direction as PI length increases. The weighting is quantitatively predicted by modelling the expected directional variance of home vectors of different lengths and assuming optimal cue integration. However, a subsequent experiment suggests ants may not actually compute an internal estimate of the PI certainty, but are using the PI home vector length as a proxy.}
    }
  • J. Xu and S. Yue, “Building up a bio-inspired visual attention model by integrating top-down shape bias and improved mean shift adaptive segmentation,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 29, iss. 4, 2015.
    [BibTeX] [Abstract] [EPrints]

    The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments. Â\copyright 2015 World Scientific Publishing Company.

    @article{lirolem20639,
              volume = {29},
              number = {4},
               month = {June},
              author = {Jiawei Xu and Shigang Yue},
               title = {Building up a bio-inspired visual attention model by integrating top-down shape bias and improved mean shift adaptive segmentation},
           publisher = {World Scientific Publishing Co. Pte Ltd},
             journal = {International Journal of Pattern Recognition and Artificial Intelligence},
                year = {2015},
            keywords = {ARRAY(0x7f7859443648)},
                 url = {http://eprints.lincoln.ac.uk/20639/},
            abstract = {The driver-assistance system (DAS) becomes quite necessary in-vehicle equipment nowadays due to the large number of road traffic accidents worldwide. An efficient DAS detecting hazardous situations robustly is key to reduce road accidents. The core of a DAS is to identify salient regions or regions of interest relevant to visual attended objects in real visual scenes for further process. In order to achieve this goal, we present a method to locate regions of interest automatically based on a novel adaptive mean shift segmentation algorithm to obtain saliency objects. In the proposed mean shift algorithm, we use adaptive Bayesian bandwidth to find the convergence of all data points by iterations and the k-nearest neighborhood queries. Experiments showed that the proposed algorithm is efficient, and yields better visual salient regions comparing with ground-truth benchmark. The proposed algorithm continuously outperformed other known visual saliency methods, generated higher precision and better recall rates, when challenged with natural scenes collected locally and one of the largest publicly available data sets. The proposed algorithm can also be extended naturally to detect moving vehicles in dynamic scenes once integrated with top-down shape biased cues, as demonstrated in our experiments. {\^A}{\copyright} 2015 World Scientific Publishing Company.}
    }
  • Z. Zhang, S. Yue, and G. Zhang, “Fly visual system inspired artificial neural network for collision detection,” Neurocomputing, vol. 153, iss. 4, pp. 221-234, 2015.
    [BibTeX] [Abstract] [EPrints]

    This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.

    @article{lirolem17881,
              volume = {153},
              number = {4},
               month = {April},
              author = {Zhuhong Zhang and Shigang Yue and Guopeng Zhang},
               title = {Fly visual system inspired artificial neural network for collision detection},
           publisher = {Elsevier},
                year = {2015},
             journal = {Neurocomputing},
               pages = {221--234},
            keywords = {ARRAY(0x7f78593b5ad0)},
                 url = {http://eprints.lincoln.ac.uk/17881/},
            abstract = {This work investigates one bio-inspired collision detection system based on fly visual neural structures, in which collision alarm is triggered if an approaching object in a direct collision course appears in the field of view of a camera or a robot, together with the relevant time region of collision. One such artificial system consists of one artificial fly visual neural network model and one collision detection mechanism. The former one is a computational model to capture membrane potentials produced by neurons. The latter one takes the outputs of the former one as its inputs, and executes three detection schemes: (i) identifying when a spike takes place through the membrane potentials and one threshold scheme; (ii) deciding the motion direction of a moving object by the Reichardt detector model; and (iii) sending collision alarms and collision regions. Experimentally, relying upon a series of video image sequences with different scenes, numerical results illustrated that the artificial system with some striking characteristics is a potentially alternative tool for collision detection.}
    }

2014

  • B. H. Amor, G. Neumann, S. Kamthe, O. Kroemer, and J. Peters, “Interaction primitives for human-robot cooperation tasks,” in 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014, pp. 2831-2837.
    [BibTeX] [Abstract] [EPrints]

    To engage in cooperative activities with human partners, robots have to possess basic interactive abilities and skills. However, programming such interactive skills is a challenging task, as each interaction partner can have different timing or an alternative way of executing movements. In this paper, we propose to learn interaction skills by observing how two humans engage in a similar task. To this end, we introduce a new representation called Interaction Primitives. Interaction primitives build on the framework of dynamic motor primitives (DMPs) by maintaining a distribution over the parameters of the DMP. With this distribution, we can learn the inherent correlations of cooperative activities which allow us to infer the behavior of the partner and to participate in the cooperation. We will provide algorithms for synchronizing and adapting the behavior of humans and robots during joint physical activities.

    @inproceedings{lirolem25773,
           booktitle = {2014 IEEE International Conference on Robotics and Automation (ICRA 2014)},
               month = {June},
               title = {Interaction primitives for human-robot cooperation tasks},
              author = {H. Ben Amor and Gerhard Neumann and S. Kamthe and O. Kroemer and J. Peters},
                year = {2014},
               pages = {2831--2837},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
            keywords = {ARRAY(0x7f78592f0b58)},
                 url = {http://eprints.lincoln.ac.uk/25773/},
            abstract = {To engage in cooperative activities with human
    partners, robots have to possess basic interactive abilities
    and skills. However, programming such interactive skills is a
    challenging task, as each interaction partner can have different
    timing or an alternative way of executing movements. In this
    paper, we propose to learn interaction skills by observing how
    two humans engage in a similar task. To this end, we introduce
    a new representation called Interaction Primitives. Interaction
    primitives build on the framework of dynamic motor primitives
    (DMPs) by maintaining a distribution over the parameters of
    the DMP. With this distribution, we can learn the inherent
    correlations of cooperative activities which allow us to infer the
    behavior of the partner and to participate in the cooperation.
    We will provide algorithms for synchronizing and adapting the
    behavior of humans and robots during joint physical activities.}
    }
  • F. Arvin, J. Murray, L. Shi, C. Zhang, and S. Yue, “Development of an autonomous micro robot for swarm robotics,” in IEEE International Conference on Mechatronics and Automation (ICMA), 2014, pp. 635-640.
    [BibTeX] [Abstract] [EPrints]

    Swarm robotic systems which are inspired from social behaviour of animals especially insects are becoming a fascinating topic for multi-robot researchers. Simulation software is mostly used for performing research in swarm robotics due the hardware complexities and cost of robot platforms. However, simulation of large numbers of these swarm robots is extremely complex and often inaccurate. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias uses a circular platform with a diameter of 4 cm. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours. The robot has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.

    @inproceedings{lirolem14837,
           booktitle = {IEEE International Conference on Mechatronics and Automation (ICMA)},
               month = {August},
               title = {Development of an autonomous micro robot for swarm robotics},
              author = {Farshad Arvin and John Murray and Licheng Shi and Chun Zhang and Shigang Yue},
           publisher = {IEEE},
                year = {2014},
               pages = {635--640},
            keywords = {ARRAY(0x7f7859451290)},
                 url = {http://eprints.lincoln.ac.uk/14837/},
            abstract = {Swarm robotic systems which are inspired from social behaviour of animals especially insects are becoming a fascinating topic for multi-robot researchers. Simulation software is mostly used for performing research in swarm robotics due the hardware complexities and cost of robot platforms. However, simulation of large numbers of these swarm robots is extremely complex and often inaccurate. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias uses a circular platform with a diameter of 4 cm. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours. The robot has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.}
    }
  • F. Arvin, J. Murray, C. Zhang, and S. Yue, “Colias: an autonomous micro robot for swarm robotic applications,” International Journal of Advanced Robotic Systems, vol. 11, iss. 113, pp. 1-10, 2014.
    [BibTeX] [Abstract] [EPrints]

    Robotic swarms that take inspiration from nature are becoming a fascinating topic for multi-robot researchers. The aim is to control a large number of simple robots enables them in order to solve common complex tasks. Due to the hardware complexities and cost of robot platforms, current research in swarm robotics is mostly performed by simulation software. Simulation of large numbers of these robots which are used in swarm robotic applications is extremely complex and often inaccurate due to poor modelling of external conditions. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias employs a circular platform with a diameter of 4 cm. It has a maximum speed of 35 cm/s that gives the ability to be used in swarm scenarios very quickly in large arenas. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours from a range of 0.5 cm to 3 m. Colias has been designed as a complete platform with supporting software development tools for robotics education and research. It has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.

    @article{lirolem14585,
              volume = {11},
              number = {113},
               month = {July},
              author = {Farshad Arvin and John Murray and Chun Zhang and Shigang Yue},
               title = {Colias: an autonomous micro robot for swarm robotic applications},
           publisher = {InTech},
                year = {2014},
             journal = {International Journal of Advanced Robotic Systems},
               pages = {1--10},
            keywords = {ARRAY(0x7f78593b6130)},
                 url = {http://eprints.lincoln.ac.uk/14585/},
            abstract = {Robotic swarms that take inspiration from nature are becoming a fascinating topic for multi-robot researchers. The aim is to control a large number of simple robots enables them in order to solve common complex tasks. Due to the hardware complexities and cost of robot platforms, current research in swarm robotics is mostly performed by simulation software. Simulation of large numbers of these robots which are used in swarm robotic applications is extremely complex and often inaccurate due to poor modelling of external conditions. In this paper we present the design of a low-cost, open-platform, autonomous micro robot (Colias) for swarm robotic applications. Colias employs a circular platform with a diameter of 4 cm. It has a maximum speed of 35 cm/s that gives the ability to be used in swarm scenarios very quickly in large arenas. Long-range infrared modules with adjustable output power allow the robot to communicate with its direct neighbours from a range of 0.5 cm to 3 m. Colias has been designed as a complete platform with supporting software development tools for robotics education and research. It has been tested in individual and swarm scenarios and the observed results demonstrate its feasibility to be used as a micro sized mobile robot as well as a low-cost platform for robot swarm applications.}
    }
  • F. Arvin, A. E. Turgut, F. Bazyari, K. B. Arikan, N. Bellotto, and S. Yue, “Cue-based aggregation with a mobile robot swarm: a novel fuzzy-based method,” Adaptive Behavior, vol. 22, iss. 3, pp. 189-206, 2014.
    [BibTeX] [Abstract] [EPrints]

    Aggregation in swarm robotics is referred to as the gathering of spatially distributed robots into a single aggregate. Aggregation can be classified as cue-based or self-organized. In cue-based aggregation, there is a cue in the environment that points to the aggregation area, whereas in self-organized aggregation no cue is present. In this paper, we proposed a novel fuzzy-based method for cue-based aggregation based on the state-of-the-art BEECLUST algorithm. In particular, we proposed three different methods: naïve, that uses a deterministic decision-making mechanism; vector-averaging, using a vectorial summation of all perceived inputs; and fuzzy, that uses a fuzzy logic controller. We used different experiment settings: one-source and two-source environments with static and dynamic conditions to compare all the methods. We observed that the fuzzy method outperformed all the other methods and it is the most robust method against noise.

    @article{lirolem13932,
              volume = {22},
              number = {3},
               month = {June},
              author = {Farshad Arvin and Ali Emre Turgut and Farhad Bazyari and Kutluk Bilge Arikan and Nicola Bellotto and Shigang Yue},
               title = {Cue-based aggregation with a mobile robot swarm: a novel fuzzy-based method},
           publisher = {Sage for International Society for Adaptive Behavior (ISAB)},
                year = {2014},
             journal = {Adaptive Behavior},
               pages = {189--206},
            keywords = {ARRAY(0x7f785944ef90)},
                 url = {http://eprints.lincoln.ac.uk/13932/},
            abstract = {Aggregation in swarm robotics is referred to as the gathering of spatially distributed robots into a single aggregate. Aggregation can be classified as cue-based or self-organized. In cue-based aggregation, there is a cue in the environment that points to the aggregation area, whereas in self-organized aggregation no cue is present. In this paper, we proposed a novel fuzzy-based method for cue-based aggregation based on the state-of-the-art BEECLUST algorithm. In particular, we proposed three different methods: na{\"i}ve, that uses a deterministic decision-making mechanism; vector-averaging, using a vectorial summation of all perceived inputs; and fuzzy, that uses a fuzzy logic controller. We used different experiment settings: one-source and two-source environments with static and dynamic conditions to compare all the methods. We observed that the fuzzy method outperformed all the other methods and it is the most robust method against noise.}
    }
  • F. Arvin, A. E. Turgut, N. Bellotto, and S. Yue, “Comparison of different cue-based swarm aggregation strategies,” in International Conference in Swarm Intelligence, 2014, pp. 1-8.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we compare different aggregation strategies for cue-based aggregation with a mobile robot swarm. We used a sound source as the cue in the environment and performed real robot and simulation based experiments. We compared the performance of two proposed aggregation algorithms we called as the vector averaging and naïve with the state-of-the-art cue-based aggregation strategy BEECLUST. We showed that the proposed strategies outperform BEECLUST method. We also illustrated the feasibility of the method in the presence of noise. The results showed that the vector averaging algorithm is more robust to noise when compared to the naïve method.

    @inproceedings{lirolem14927,
               month = {October},
              author = {Farshad Arvin and Ali Emre Turgut and Nicola Bellotto and Shigang Yue},
                note = {Proceedings, Part I, series volume 8794},
           booktitle = {International Conference in Swarm Intelligence},
               title = {Comparison of different cue-based swarm aggregation strategies},
           publisher = {Springer},
               pages = {1--8},
                year = {2014},
            keywords = {ARRAY(0x7f7858fa0538)},
                 url = {http://eprints.lincoln.ac.uk/14927/},
            abstract = {In this paper, we compare different aggregation strategies for cue-based aggregation with a mobile robot swarm. We used a sound source as the cue in the environment and performed real robot and simulation based experiments. We compared the performance of two proposed aggregation algorithms we called as the vector averaging and na{\"i}ve with the state-of-the-art cue-based aggregation strategy BEECLUST. We showed that the proposed strategies outperform BEECLUST method. We also illustrated the feasibility of the method in the presence of noise. The results showed that the vector averaging algorithm is more robust to noise when compared to the na{\"i}ve method.}
    }
  • A. Attar, X. Xie, C. Zhang, Z. Wang, and S. Yue, “Wireless Micro-Ball endoscopic image enhancement using histogram information,” in Conference proceedings of the 2014 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Institute of Electrical and Electronics Engineers Inc., 2014, pp. 3337-3340.
    [BibTeX] [Abstract] [EPrints]

    Wireless endoscopy systems is a new innovative method widely used for gastrointestinal tract examination in recent decade. Wireless Micro-Ball endoscopy system with multiple image sensors is the newest proposed method which can make a full view image of the gastrointestinal tract. But still the quality of images from this new wireless endoscopy system is not satisfactory. It’s hard for doctors and specialist to easily examine and interpret the captured images. The image features also are not distinct enough to be used for further processing. So as to enhance these low-contrast endoscopic images a new image enhancement method based on the endoscopic images features and color distribution is proposed in this work. The enhancement method is performed on three main steps namely color space transformation, edge preserving mask formation, and histogram information correction. The luminance component of CIE Lab, YCbCr, and HSV color space is enhanced in this method and then two other components added finally to form an enhanced color image. The experimental result clearly show the robustness of the method. \copyright 2014 IEEE.

    @incollection{lirolem17582,
               month = {August},
              author = {Abdolrahman Attar and Xiang Xie and Chun Zhang and Zhihua Wang and Shigang Yue},
                note = {Conference of 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014 ; Conference Date: 26 - 30 August 2014; Chicago, USA  Conference Code:109045},
           booktitle = {Conference proceedings of the 2014 Annual International Conference of the IEEE Engineering in Medicine and Biology Society},
               title = {Wireless Micro-Ball endoscopic image enhancement using histogram information},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
                year = {2014},
             journal = {2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBC 2014},
               pages = {3337--3340},
            keywords = {ARRAY(0x7f7859409e38)},
                 url = {http://eprints.lincoln.ac.uk/17582/},
            abstract = {Wireless endoscopy systems is a new innovative method widely used for gastrointestinal tract examination in recent decade. Wireless Micro-Ball endoscopy system with multiple image sensors is the newest proposed method which can make a full view image of the gastrointestinal tract. But still the quality of images from this new wireless endoscopy system is not satisfactory. It's hard for doctors and specialist to easily examine and interpret the captured images. The image features also are not distinct enough to be used for further processing. So as to enhance these low-contrast endoscopic images a new image enhancement method based on the endoscopic images features and color distribution is proposed in this work. The enhancement method is performed on three main steps namely color space transformation, edge preserving mask formation, and histogram information correction. The luminance component of CIE Lab, YCbCr, and HSV color space is enhanced in this method and then two other components added finally to form an enhanced color image. The experimental result clearly show the robustness of the method. {\copyright} 2014 IEEE.}
    }
  • M. Barnes, “Computer vision based detection and identification of potato blemishes,” PhD Thesis, 2014.
    [BibTeX] [Abstract] [EPrints]

    .

    @phdthesis{lirolem14568,
               month = {July},
               title = {Computer vision based detection and identification of potato blemishes},
              school = {University of Lincoln},
              author = {Michael Barnes},
                year = {2014},
            keywords = {ARRAY(0x7f78593b60d0)},
                 url = {http://eprints.lincoln.ac.uk/14568/},
            abstract = {.}
    }
  • A. Cheung, M. Collett, T. S. Collett, A. Dewar, F. Dyer, P. Graham, M. Mangan, A. Narendra, A. Philippides, W. Stürzl, B. Webb, A. Wystrach, and J. Zeil, “Still no convincing evidence for cognitive map use by honeybees,” Proceedings of the National Academy of Sciences, vol. 111, iss. 42, p. E4396–E4397, 2014.
    [BibTeX] [Abstract] [EPrints]

    Cheeseman et al. (1) claim that an ability of honey bees to travel home through a landscape with conflicting information from a celestial compass proves the bees’ use of a cognitive map. Their claim involves a curious assumption about the visual information that can be extracted from the terrain: that there is sufficient information for a bee to identify where it is, but insufficient to guide its path without resorting to a cognitive map. We contend that the authors? claims are unfounded.

    @article{lirolem23584,
              volume = {111},
              number = {42},
               month = {October},
              author = {Allen Cheung and Matthew Collett and Thomas S. Collett and Alex Dewar and Fred Dyer and Paul Graham and Michael Mangan and Ajay Narendra and Andrew Philippides and Wolfgang St{\"u}rzl and Barbara Webb and Antoine Wystrach and Jochen Zeil},
               title = {Still no convincing evidence for cognitive map use by honeybees},
           publisher = {National Academy of Sciences},
                year = {2014},
             journal = {Proceedings of the National Academy of Sciences},
               pages = {E4396--E4397},
            keywords = {ARRAY(0x7f78592c8d78)},
                 url = {http://eprints.lincoln.ac.uk/23584/},
            abstract = {Cheeseman et al. (1) claim that an ability of honey bees to travel home through a landscape with conflicting information from a celestial compass proves the bees' use of a cognitive map. Their claim involves a curious assumption about the visual information that can be extracted from the terrain: that there is sufficient information for a bee to identify where it is, but insufficient to guide its path without resorting to a cognitive map. We contend that the authors? claims are unfounded.}
    }
  • A. Colome, G. Neumann, J. Peters, and C. Torras, “Dimensionality reduction for probabilistic movement primitives,” in Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, 2014, pp. 794-800.
    [BibTeX] [Abstract] [EPrints]

    Humans as well as humanoid robots can use a large number of degrees of freedom to solve very complex motor tasks. The high-dimensionality of these motor tasks adds difficulties to the control problem and machine learning algorithms. However, it is well known that the intrinsic dimensionality of many human movements is small in comparison to the number of employed DoFs, and hence, the movements can be represented by a small number of synergies encoding the couplings between DoFs. In this paper, we want to apply Dimensionality Reduction (DR) to a recent movement representation used in robotics, called Probabilistic Movement Primitives (ProMP). While ProMP have been shown to have many benefits, they suffer with the high-dimensionality of a robotic system as the number of parameters of a ProMP scales quadratically with the dimensionality. We use probablistic dimensionality reduction techniques based on expectation maximization to extract the unknown synergies from a given set of demonstrations. The ProMP representation is now estimated in the low-dimensional space of the synergies. We show that our dimensionality reduction is more efficient both for encoding a trajectory from data and for applying Reinforcement Learning with Relative Entropy Policy Search (REPS).

    @inproceedings{lirolem25756,
              volume = {2015-F},
               month = {November},
              author = {A. Colome and G. Neumann and J. Peters and C. Torras},
           booktitle = {Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on},
               title = {Dimensionality reduction for probabilistic movement primitives},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {794--800},
                year = {2014},
            keywords = {ARRAY(0x7f7859120a90)},
                 url = {http://eprints.lincoln.ac.uk/25756/},
            abstract = {Humans as well as humanoid robots can use a large number of degrees of freedom to solve very complex motor tasks. The high-dimensionality of these motor tasks adds difficulties to the control problem and machine learning algorithms. However, it is well known that the intrinsic dimensionality of many human movements is small in comparison to the number of employed DoFs, and hence, the movements can be represented by a small number of synergies encoding the couplings between DoFs. In this paper, we want to apply Dimensionality Reduction (DR) to a recent movement representation used in robotics, called Probabilistic Movement Primitives (ProMP). While ProMP have been shown to have many benefits, they suffer with the high-dimensionality of a robotic system as the number of parameters of a ProMP scales quadratically with the dimensionality. We use probablistic dimensionality reduction techniques based on expectation maximization to extract the unknown synergies from a given set of demonstrations. The ProMP representation is now estimated in the low-dimensional space of the synergies. We show that our dimensionality reduction is more efficient both for encoding a trajectory from data and for applying Reinforcement Learning with Relative Entropy Policy Search (REPS).}
    }
  • H. Cuayahuitl, I. Kruijff-Korbayová, and N. Dethlefs, “Nonstrict hierarchical reinforcement learning for interactive systems and robots,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 4, iss. 3, p. 15, 2014.
    [BibTeX] [Abstract] [EPrints]

    Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.

    @article{lirolem22211,
              volume = {4},
              number = {3},
               month = {October},
              author = {Heriberto Cuayahuitl and Ivana Kruijff-Korbayov{\'a} and Nina Dethlefs},
               title = {Nonstrict hierarchical reinforcement learning for interactive systems and robots},
           publisher = {Association for Computing Machinery (ACM)},
                year = {2014},
             journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
               pages = {15},
            keywords = {ARRAY(0x7f78593b5ce0)},
                 url = {http://eprints.lincoln.ac.uk/22211/},
            abstract = {Conversational systems and robots that use reinforcement learning for policy optimization in large domains often face the problem of limited scalability. This problem has been addressed either by using function approximation techniques that estimate the approximate true value function of a policy or by using a hierarchical decomposition of a learning task into subtasks. We present a novel approach for dialogue policy optimization that combines the benefits of both hierarchical control and function approximation and that allows flexible transitions between dialogue subtasks to give human users more control over the dialogue. To this end, each reinforcement learning agent in the hierarchy is extended with a subtask transition function and a dynamic state space to allow flexible switching between subdialogues. In addition, the subtask policies are represented with linear function approximation in order to generalize the decision making to situations unseen in training. Our proposed approach is evaluated in an interactive conversational robot that learns to play quiz games. Experimental results, using simulation and real users, provide evidence that our proposed approach can lead to more flexible (natural) interactions than strict hierarchical control and that it is preferred by human users.}
    }
  • H. Cuayahuitl, L. Frommberger, N. Dethlefs, A. Raux, M. Marge, and H. Zender, “Introduction to the special issue on Machine learning for multiple modalities in interactive systems and robots,” ACM Transactions on Interactive Intelligent Systems (TiiS), vol. 4, iss. 3, p. 12e, 2014.
    [BibTeX] [Abstract] [EPrints]

    This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.

    @article{lirolem22212,
              volume = {4},
              number = {3},
               month = {October},
              author = {Heriberto Cuayahuitl and Lutz Frommberger and Nina Dethlefs and Antoine Raux and Mathew Marge and Hendrik Zender},
               title = {Introduction to the special issue on Machine learning for multiple modalities in interactive systems and robots},
           publisher = {Association for Computing Machinery (ACM)},
                year = {2014},
             journal = {ACM Transactions on Interactive Intelligent Systems (TiiS)},
               pages = {12e},
            keywords = {ARRAY(0x7f78592c9138)},
                 url = {http://eprints.lincoln.ac.uk/22212/},
            abstract = {This special issue highlights research articles that apply machine learning to robots and other systems that interact with users through more than one modality, such as speech, gestures, and vision. For example, a robot may coordinate its speech with its actions, taking into account (audio-)visual feedback during their execution. Machine learning provides interactive systems with opportunities to improve performance not only of individual components but also of the system as a whole. However, machine learning methods that encompass multiple modalities of an interactive system are still relatively hard to find. The articles in this special issue represent examples that contribute to filling this gap.}
    }
  • C. Dann, G. Neumann, and J. Peters, “Policy evaluation with temporal differences: a survey and comparison,” Journal of Machine Learning Research, vol. 15, pp. 809-883, 2014.
    [BibTeX] [Abstract] [EPrints]

    Policy evaluation is an essential step in most reinforcement learning approaches. It yields a value function, the quality assessment of states for a given policy, which can be used in a policy improvement step. Since the late 1980s, this research area has been dominated by temporal-difference (TD) methods due to their data-efficiency. However, core issues such as stability guarantees in the off-policy scenario, improved sample efficiency and probabilistic treatment of the uncertainty in the estimates have only been tackled recently, which has led to a large number of new approaches. This paper aims at making these new developments accessible in a concise overview, with foci on underlying cost functions, the off-policy scenario as well as on regularization in high dimensional feature spaces. By presenting the first extensive, systematic comparative evaluations comparing TD, LSTD, LSPE, FPKF, the residual- gradient algorithm, Bellman residual minimization, GTD, GTD2 and TDC, we shed light on the strengths and weaknesses of the methods. Moreover, we present alternative versions of LSTD and LSPE with drastically improved off-policy performance.

    @article{lirolem25768,
              volume = {15},
               month = {March},
              author = {C. Dann and G. Neumann and J. Peters},
               title = {Policy evaluation with temporal differences: a survey and comparison},
           publisher = {Massachusetts Institute of Technology Press (MIT Press) / Microtome Publishing},
             journal = {Journal of Machine Learning Research},
               pages = {809--883},
                year = {2014},
            keywords = {ARRAY(0x7f7859404df8)},
                 url = {http://eprints.lincoln.ac.uk/25768/},
            abstract = {Policy evaluation is an essential step in most reinforcement learning approaches. It yields a value function, the quality assessment of states for a given policy, which can be used in a policy improvement step. Since the late 1980s, this research area has been dominated by temporal-difference (TD) methods due to their data-efficiency. However, core issues such as stability guarantees in the off-policy scenario, improved sample efficiency and probabilistic treatment of the uncertainty in the estimates have only been tackled recently, which has led to a large number of new approaches.
    
    This paper aims at making these new developments accessible in a concise overview, with foci on underlying cost functions, the off-policy scenario as well as on regularization in high dimensional feature spaces. By presenting the first extensive, systematic comparative evaluations comparing TD, LSTD, LSPE, FPKF, the residual- gradient algorithm, Bellman residual minimization, GTD, GTD2 and TDC, we shed light on the strengths and weaknesses of the methods. Moreover, we present alternative versions of LSTD and LSPE with drastically improved off-policy performance.}
    }
  • N. Dethlefs and H. Cuayahuitl, “Hierarchical reinforcement learning for situated language generation,” Natural Language Engineering, vol. 21, iss. 3, pp. 391-435, 2014.
    [BibTeX] [Abstract] [EPrints]

    Natural Language Generation systems in interactive settings often face a multitude of choices, given that the communicative effect of each utterance they generate depends crucially on the interplay between its physical circumstances, addressee and interaction history. This is particularly true in interactive and situated settings. In this paper we present a novel approach for situated Natural Language Generation in dialogue that is based on hierarchical reinforcement learning and learns the best utterance for a context by optimisation through trial and error. The model is trained from human?human corpus data and learns particularly to balance the trade-off between efficiency and detail in giving instructions: the user needs to be given sufficient information to execute their task, but without exceeding their cognitive load. We present results from simulation and a task-based human evaluation study comparing two different versions of hierarchical reinforcement learning: One operates using a hierarchy of policies with a large state space and local knowledge, and the other additionally shares knowledge across generation subtasks to enhance performance. Results show that sharing knowledge across subtasks achieves better performance than learning in isolation, leading to smoother and more successful interactions that are better perceived by human users.

    @article{lirolem22213,
              volume = {21},
              number = {3},
               month = {May},
              author = {Nina Dethlefs and Heriberto Cuayahuitl},
               title = {Hierarchical reinforcement learning for situated language generation},
           publisher = {Cambridge University Press},
                year = {2014},
             journal = {Natural Language Engineering},
               pages = {391--435},
            keywords = {ARRAY(0x7f7859462548)},
                 url = {http://eprints.lincoln.ac.uk/22213/},
            abstract = {Natural Language Generation systems in interactive settings often face a multitude of choices, given that the communicative effect of each utterance they generate depends crucially on the interplay between its physical circumstances, addressee and interaction history. This is particularly true in interactive and situated settings. In this paper we present a novel approach for situated Natural Language Generation in dialogue that is based on hierarchical reinforcement learning and learns the best utterance for a context by optimisation through trial and error. The model is trained from human?human corpus data and learns particularly to balance the trade-off between efficiency and detail in giving instructions: the user needs to be given sufficient information to execute their task, but without exceeding their cognitive load. We present results from simulation and a task-based human evaluation study comparing two different versions of hierarchical reinforcement learning: One operates using a hierarchy of policies with a large state space and local knowledge, and the other additionally shares knowledge across generation subtasks to enhance performance. Results show that sharing knowledge across subtasks achieves better performance than learning in isolation, leading to smoother and more successful interactions that are better perceived by human users.}
    }
  • C. Dondrup, N. Bellotto, and M. Hanheide, “Social distance augmented qualitative trajectory calculus for human-robot spatial interaction,” in Robot and Human Interactive Communication, 2014 RO-MAN, 2014, pp. 519-524.
    [BibTeX] [Abstract] [EPrints]

    In this paper we propose to augment a wellestablished Qualitative Trajectory Calculus (QTC) by incorporating social distances into the model to facilitate a richer and more powerful representation of Human-Robot Spatial Interaction (HRSI). By combining two variants of QTC that implement different resolutions and switching between them based on distance thresholds we show that we are able to both reduce the complexity of the representation and at the same time enrich QTC with one of the core HRSI concepts: proxemics. Building on this novel integrated QTC model, we propose to represent the joint spatial behaviour of a human and a robot employing a probabilistic representation based on Hidden Markov Models. We show the appropriateness of our approach by encoding different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to represent and classify these behaviours using social distance-augmented QTC.

    @inproceedings{lirolem15832,
           booktitle = {Robot and Human Interactive Communication, 2014 RO-MAN},
               month = {October},
               title = {Social distance augmented qualitative trajectory calculus for human-robot spatial interaction},
              author = {Christian Dondrup and Nicola Bellotto and Marc Hanheide},
           publisher = {IEEE},
                year = {2014},
               pages = {519--524},
            keywords = {ARRAY(0x7f78592c6ce0)},
                 url = {http://eprints.lincoln.ac.uk/15832/},
            abstract = {In this paper we propose to augment a wellestablished Qualitative Trajectory Calculus (QTC) by incorporating social distances into the model to facilitate a richer and more powerful representation of Human-Robot Spatial Interaction (HRSI). By combining two variants of QTC that implement different resolutions and switching between them based on distance thresholds we show that we are able to both reduce the complexity of the representation and at the same time enrich QTC with one of the core HRSI concepts: proxemics. Building on this novel integrated QTC model, we propose to represent the joint spatial behaviour of a human and a robot employing a probabilistic representation based on Hidden Markov Models. We show the appropriateness of our approach by encoding different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to represent and classify these behaviours using  social distance-augmented QTC.}
    }
  • C. Dondrup, M. Hanheide, and N. Bellotto, “A probabilistic model of human-robot spatial interaction using a qualitative trajectory calculus,” in AAAI Spring Symposium: "Qualitative Representations for Robots", 2014.
    [BibTeX] [Abstract] [EPrints]

    In this paper we propose a probabilistic model for Human-Robot Spatial Interaction (HRSI) using a Qualitative Trajectory Calculus (QTC). In particular, we will build on previous work representing HRSI as a Markov chain of QTC states and evolve this to an approach using a Hidden Markov Model representation. Our model accounts for the invalidity of certain transitions within the QTC to reduce the complexity of the probabilistic model and to ensure state sequences in accordance to this representational framework. We show the appropriateness of our approach by using the probabilistic model to encode different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to classify these behaviours reliably. Copyright Â\copyright 2014, Association for the Advancement of Artificial Intelligence. All rights reserved.

    @inproceedings{lirolem13523,
           booktitle = {AAAI Spring Symposium: "Qualitative Representations for Robots"},
               month = {March},
               title = {A probabilistic model of human-robot spatial interaction using a qualitative trajectory calculus},
              author = {Christian Dondrup and Marc Hanheide and Nicola Bellotto},
           publisher = {AAAI / AI Access Foundation},
                year = {2014},
            keywords = {ARRAY(0x7f785946cac8)},
                 url = {http://eprints.lincoln.ac.uk/13523/},
            abstract = {In this paper we propose a probabilistic model for Human-Robot Spatial Interaction (HRSI) using a Qualitative Trajectory Calculus (QTC). In particular, we will build on previous work representing HRSI as a Markov chain of QTC states and evolve this to an approach using a Hidden Markov Model representation. Our model accounts for the invalidity of certain transitions within the QTC to reduce the complexity of the probabilistic model and to ensure state sequences in accordance to this representational framework. We show the appropriateness of our approach by using the probabilistic model to encode different HRSI behaviours observed in a human-robot interaction study and show how the models can be used to classify these behaviours reliably. Copyright {\^A}{\copyright} 2014, Association for the Advancement of Artificial Intelligence. All rights reserved.}
    }
  • C. Dondrup, C. Lichtenthaeler, and M. Hanheide, “Hesitation signals in human-robot head-on encounters: a pilot study,” in 9th ACM/IEEE International Conference on Human Robot Interaction, 2014, pp. 154-155.
    [BibTeX] [Abstract] [EPrints]

    The motivation for this research stems from the future vision of being able to buy a mobile service robot for your own household, unpack it, switch it on, and have it behave in an intelligent way; but of course it also has to adapt to your personal preferences over time. My work is focusing on the spatial aspect of the robot?s behaviours, which means when it is moving in a confined, shared space with a human it will also take the communicative character of these movements into account. This adaptation to the users preferences should come from experience which the robot gathers throughout several days or months of interaction and not from a programmer hard-coding certain behaviours

    @inproceedings{lirolem13570,
           booktitle = {9th ACM/IEEE International Conference on Human Robot Interaction},
               month = {March},
               title = {Hesitation signals in human-robot head-on encounters: a pilot study},
              author = {Christian Dondrup and Christina Lichtenthaeler and Marc Hanheide},
           publisher = {IEEE},
                year = {2014},
               pages = {154--155},
            keywords = {ARRAY(0x7f78594025c0)},
                 url = {http://eprints.lincoln.ac.uk/13570/},
            abstract = {The motivation for this research stems from the future vision of being able to buy a mobile service robot for your own household, unpack it, switch it on, and have it behave in an intelligent way; but of course it also has to adapt to your personal preferences over time. My work is focusing on the spatial aspect of the robot?s behaviours, which means when it is moving in a confined, shared space with a human it will also take the communicative character of these movements into account. This adaptation to the users preferences should come from experience which the robot gathers throughout several days or months of interaction and not from a programmer hard-coding certain behaviours}
    }
  • T. Duckett and T. Krajnik, “A frequency-based approach to long-term robotic mapping,” in ICRA 2014 Workshop on Long Term Autonomy, 2014.
    [BibTeX] [Abstract] [EPrints]

    While mapping of static environments has been widely studied, long-term mapping in non-stationary environments is still an open problem. In this talk, we present a novel approach for long-term representation of populated environments, where many of the observed changes are caused by humans performing their daily activities. We propose to model the environment’s dynamics by its frequency spectrum, as a combination of harmonic functions that correspond to periodic processes influencing the environment. Such a representation not only allows representation of environment dynamics over arbitrary timescales with constant memory requirements, but also prediction of future environment states. The proposed approach can be applied to many of the state-of-the-art environment models. In particular, we show that occupancy grids, topological or landmark maps can be easily extended to represent dynamic environments. We present experiments using data collected by a mobile robot patrolling an indoor environment over a period of one month, where frequency-enhanced models were compared to their static counterparts in four scenarios: i) 3D map building, ii) environment state prediction, iii) topological localisation and iv) anomaly detection, in order to verify the model’s ability to detect unusual events. In all these cases, the frequency-enhanced models outperformed their static counterparts.

    @inproceedings{lirolem14422,
           booktitle = {ICRA 2014 Workshop on Long Term Autonomy},
               month = {June},
               title = {A frequency-based approach to long-term robotic mapping},
              author = {Tom Duckett and Tomas Krajnik},
                year = {2014},
            keywords = {ARRAY(0x7f78593b5ba8)},
                 url = {http://eprints.lincoln.ac.uk/14422/},
            abstract = {While mapping of static environments has been widely studied, long-term mapping in non-stationary environments is still an open problem. In this talk, we present a novel approach for long-term representation of populated environments, where many of the observed changes are caused by humans performing their daily activities. We propose to model the environment's dynamics by its frequency spectrum, as a combination of harmonic functions that correspond to periodic processes influencing the environment. Such a representation not only allows representation of environment dynamics over arbitrary timescales with constant memory requirements, but also prediction of future environment states. The proposed approach can be applied to many of the state-of-the-art environment models. In particular, we show that occupancy grids, topological or landmark maps can be easily extended to represent dynamic environments. We present experiments using data collected by a mobile robot patrolling an indoor environment over a period of one month, where frequency-enhanced models were compared to their static counterparts in four scenarios: i) 3D map building, ii) environment state prediction, iii) topological localisation and iv) anomaly detection, in order to verify the model's ability to detect unusual events. In all these cases, the frequency-enhanced models outperformed their static counterparts.}
    }
  • V. Gomez, H. J. Kappen, J. Peters, and G. Neumann, “Policy search for path integral control,” in Machine Learning and Knowledge Discovery in Databases – European Conference, ECML/PKDD 2014, 2014, pp. 482-497.
    [BibTeX] [Abstract] [EPrints]

    Path integral (PI) control defines a general class of control problems for which the optimal control computation is equivalent to an inference problem that can be solved by evaluation of a path integral over state trajectories. However, this potential is mostly unused in real-world problems because of two main limitations: first, current approaches can typically only be applied to learn open-loop controllers and second, current sampling procedures are inefficient and not scalable to high dimensional systems. We introduce the efficient Path Integral Relative-Entropy Policy Search (PI-REPS) algorithm for learning feedback policies with PI control. Our algorithm is inspired by information theoretic policy updates that are often used in policy search. We use these updates to approximate the state trajectory distribution that is known to be optimal from the PI control theory. Our approach allows for a principled treatment of different sampling distributions and can be used to estimate many types of parametric or non-parametric feedback controllers. We show that PI-REPS significantly outperforms current methods and is able to solve tasks that are out of reach for current methods.

    @inproceedings{lirolem25770,
              volume = {8724 L},
              number = {PART 1},
              author = {Vincenc Gomez and Hilbert J. Kappen and Jan Peters and Gerhard Neumann},
           booktitle = {Machine Learning and Knowledge Discovery in Databases - European Conference, ECML/PKDD 2014},
               title = {Policy search for path integral control},
           publisher = {Springer},
                year = {2014},
             journal = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)},
               pages = {482--497},
            keywords = {ARRAY(0x7f78593aef08)},
                 url = {http://eprints.lincoln.ac.uk/25770/},
            abstract = {Path integral (PI) control defines a general class of control problems for which the optimal control computation is equivalent to an inference problem that can be solved by evaluation of a path integral over state trajectories. However, this potential is mostly unused in real-world problems because of two main limitations: first, current approaches can typically only be applied to learn open-loop controllers and second, current sampling procedures are inefficient and not scalable to high dimensional systems. We introduce the efficient Path Integral Relative-Entropy Policy Search (PI-REPS) algorithm for learning feedback policies with PI control. Our algorithm is inspired by information theoretic policy updates that are often used in policy search. We use these updates to approximate the state trajectory distribution that is known to be optimal from the PI control theory. Our approach allows for a principled treatment of different sampling distributions and can be used to estimate many types of parametric or non-parametric feedback controllers. We show that PI-REPS significantly outperforms current methods and is able to solve tasks that are out of reach for current methods.}
    }
  • C. Hu, F. Arvin, and S. Yue, “Development of a bio-inspired vision system for mobile micro-robots,” in IEEE International Conferences on Development and Learning and Epigenetic Robotics (ICDL-Epirob), 2014, pp. 81-86.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a new bio-inspired vision system for mobile micro-robots. The processing method takes inspiration from vision of locusts in detecting the fast approaching objects. Research suggested that locusts use wide field visual neuron called the lobula giant movement detector to respond to imminent collisions. We employed the locusts’ vision mechanism to motion control of a mobile robot. The selected image processing method is implemented on a developed extension module using a low-cost and fast ARM processor. The vision module is placed on top of a micro-robot to control its trajectory and to avoid obstacles. The observed results from several performed experiments demonstrated that the developed extension module and the inspired vision system are feasible to employ as a vision module for obstacle avoidance and motion control.

    @inproceedings{lirolem16334,
           booktitle = {IEEE International Conferences on Development and Learning and Epigenetic Robotics (ICDL-Epirob)},
               month = {October},
               title = {Development of a bio-inspired vision system for mobile micro-robots},
              author = {Cheng Hu and Farshad Arvin and Shigang Yue},
           publisher = {IEEE},
                year = {2014},
               pages = {81--86},
            keywords = {ARRAY(0x7f78592ea040)},
                 url = {http://eprints.lincoln.ac.uk/16334/},
            abstract = {In this paper, we present a new bio-inspired vision system for mobile micro-robots. The processing method takes inspiration from vision of locusts in detecting the fast approaching objects. Research suggested that locusts use wide field visual neuron called the lobula giant movement detector to respond to imminent collisions. We employed the locusts' vision mechanism to motion control of a mobile robot. The selected image processing method is implemented on a developed extension module using a low-cost and fast ARM processor. The vision module is placed on top of a micro-robot to control its trajectory and to avoid obstacles. The observed results from several performed experiments demonstrated that the developed extension module and the inspired vision system are feasible to employ as a vision module for obstacle avoidance and motion control.}
    }
  • K. Iliopoulos, N. Bellotto, and N. Mavridis, “From sequence to trajectory and vice versa: solving the inverse QTC problem and coping with real-world trajectories,” in AAAI Spring Symposium: "Qualitative Representations for Robots", 2014.
    [BibTeX] [Abstract] [EPrints]

    Spatial interactions between agents carry information of high value to human observers, as exemplified by the high-level interpretations that humans make when watching the Heider and Simmel movie, or other such videos which just contain motions of simple objects, such as points, lines and triangles. However, not all the information contained in a pair of continuous trajectories is important; and thus the need for qualitative descriptions of interaction trajectories arises. Towards that purpose, Qualitative Trajectory Calculus (QTC) has been proposed in (Van de Weghe, 2004). However, the original definition of QTC handles uncorrupted continuous-time trajectories, while real-world signals are noisy and sampled in discrete-time. Also, although QTC presents a method for transforming trajectories to qualitative descriptions, the inverse problem has not yet been studied. Thus, in this paper, after discussing several aspects of the transition from ideal QTC to discrete-time noisy QTC, we introduce a novel algorithm for solving the QTC inverse problem; i.e. transforming qualitative descriptions to archetypal trajectories that satisfy them. Both of these problems are particularly important for the successful application of qualitative trajectory calculus to Human-Robot Interaction.

    @inproceedings{lirolem13519,
           booktitle = {AAAI Spring Symposium: "Qualitative Representations for Robots"},
               month = {March},
               title = {From sequence to trajectory and vice versa: solving the inverse QTC problem and coping with real-world trajectories},
              author = {Konstantinos Iliopoulos and Nicola Bellotto and Nikolaos Mavridis},
           publisher = {AAAI},
                year = {2014},
            keywords = {ARRAY(0x7f785940bd88)},
                 url = {http://eprints.lincoln.ac.uk/13519/},
            abstract = {Spatial interactions between agents carry information of high value to human observers, as exemplified by the high-level interpretations that humans make when watching the Heider and Simmel movie, or other such videos which just contain motions of simple objects, such as points, lines and triangles. However, not all the information contained in a pair of continuous trajectories is important; and thus the need for qualitative descriptions of interaction trajectories arises. Towards that purpose, Qualitative Trajectory Calculus (QTC) has been proposed in (Van de Weghe, 2004). However, the original definition of QTC handles uncorrupted continuous-time trajectories, while real-world signals are noisy and sampled in discrete-time. Also, although QTC presents a method for transforming trajectories to qualitative descriptions, the inverse problem has not yet been studied. Thus, in this paper, after discussing several aspects of the transition from ideal QTC to discrete-time noisy QTC, we introduce a novel algorithm for solving the QTC inverse problem; i.e. transforming qualitative descriptions to archetypal trajectories that satisfy them. Both of these problems are particularly important for the successful application of qualitative trajectory calculus to Human-Robot Interaction.}
    }
  • T. Krajnik, J. P. Fentanes, G. Cielniak, C. Dondrup, and T. Duckett, “Spectral analysis for long-term robotic mapping,” in 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), 2014.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a new approach to mobile robot mapping in long-term scenarios. So far, the environment models used in mobile robotics have been tailored to capture static scenes and dealt with the environment changes by means of ?memory decay?. While these models keep up with slowly changing environments, their utilization in dynamic, real world environments is difficult. The representation proposed in this paper models the environment?s spatio-temporal dynamics by its frequency spectrum. The spectral representation of the time domain allows to identify, analyse and remember regularly occurring environment processes in a computationally efficient way. Knowledge of the periodicity of the different environment processes constitutes the model predictive capabilities, which are especially useful for long-term mobile robotics scenarios. In the experiments presented, the proposed approach is applied to data collected by a mobile robot patrolling an indoor environment over a period of one week. Three scenarios are investigated, including intruder detection and 4D mapping. The results indicate that the proposed method allows to represent arbitrary timescales with constant (and low) memory requirements, achieving compression rates up to 106 . Moreover, the representation allows for prediction of future environment?s state with $\sim$ 90\% precision.

    @inproceedings{lirolem13273,
           booktitle = {2014 IEEE International Conference on Robotics and Automation (ICRA 2014)},
               month = {May},
               title = {Spectral analysis for long-term robotic mapping},
              author = {Tomas Krajnik and Jaime Pulido Fentanes and Grzegorz Cielniak and Christian Dondrup and Tom Duckett},
           publisher = {IEEE},
                year = {2014},
            keywords = {ARRAY(0x7f785940dba0)},
                 url = {http://eprints.lincoln.ac.uk/13273/},
            abstract = {This paper presents a new approach to mobile robot mapping in long-term scenarios. So far, the environment models used in mobile robotics have been tailored to capture static scenes and dealt with the environment changes by means of ?memory decay?. While these models keep up with slowly changing environments, their utilization in dynamic, real world
    environments is difficult.
    
    The representation proposed in this paper models the environment?s spatio-temporal dynamics by its frequency spectrum. The spectral representation of the time domain allows to identify, analyse and remember regularly occurring environment processes in a computationally efficient way. Knowledge of the periodicity of the different environment processes constitutes the model predictive capabilities, which are especially useful for long-term mobile robotics scenarios.
    
    In the experiments presented, the proposed approach is applied to data collected by a mobile robot patrolling an indoor
    environment over a period of one week. Three scenarios are investigated, including intruder detection and 4D mapping. The results indicate that the proposed method allows to represent arbitrary timescales with constant (and low) memory requirements, achieving compression rates up to 106 . Moreover, the representation allows for prediction of future environment?s state with {$\sim$} 90\% precision.}
    }
  • T. Krajnik, N. Matias, J. Faigl, P. Vanek, M. Saska, L. Preucil, T. Duckett, and M. Marta, “A practical multirobot localization system,” Journal of Intelligent and Robotic Systems, vol. 76, iss. 3-4, pp. 539-562, 2014.
    [BibTeX] [Abstract] [EPrints]

    We present a fast and precise vision-based software intended for multiple robot localization. The core component of the software is a novel and efficient algorithm for black and white pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost cameras, the core algorithm is able to process hundreds of images per second while tracking hundreds of objects with a millimeter precision. In addition, we present the method’s mathematical model, which allows to estimate the expected localization precision, area of coverage, and processing speed from the camera’s intrinsic parameters and hardware’s processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions is verified in several experiments. Apart from the method description, we also make its source code public at $\backslash$emph\http://purl.org/robotics/whycon\; so, it can be used as an enabling technology for various mobile robotic problems.

    @article{lirolem13653,
              volume = {76},
              number = {3-4},
               month = {December},
              author = {Tomas Krajnik and Nitsche Matias and Jan Faigl and Petr Vanek and Martin Saska and Libor Preucil and Tom Duckett and Mejail Marta},
               title = {A practical multirobot localization system},
           publisher = {Springer Heidelberg},
                year = {2014},
             journal = {Journal of Intelligent and Robotic Systems},
               pages = {539--562},
            keywords = {ARRAY(0x7f785941a440)},
                 url = {http://eprints.lincoln.ac.uk/13653/},
            abstract = {We present a fast and precise vision-based software intended for multiple robot localization. The core component of the software is a novel and efficient algorithm for black and white pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost cameras, the core algorithm is able to process hundreds of images per second while tracking hundreds of objects with a millimeter precision. In addition, we present the method's mathematical model, which allows to estimate the expected localization precision, area of coverage, and processing speed from the camera's intrinsic parameters and hardware's processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions is verified in several experiments.  Apart from the method description, we also make its source code public at {$\backslash$}emph\{http://purl.org/robotics/whycon\}; so, it can be used as an enabling technology for various mobile robotic problems.}
    }
  • T. Krajnik, J. Santos, B. Seemann, and T. Duckett, “FROctomap: an efficient spatio-temporal environment representation,” in Advances in Autonomous Robotics Systems, M. Mistry, A. Leonardis, and M. Witkowski, Eds., Springer International Publishing, 2014, vol. 8717, pp. 281-282.
    [BibTeX] [Abstract] [EPrints]

    We present a novel software tool intended for mobile robot mapping in long-term scenarios. The method allows for efficient volumetric representation of dynamic three-dimensional environments over long periods of time. It is based on a combination of a well-established 3D mapping framework called Octomaps and an idea to model environment dynamics by its frequency spectrum. The proposed method allows not only for efficient representation, but also reliable prediction of the future states of dynamic three-dimensional environments. Our spatio-temporal mapping framework is available as an open-source C++ library and a ROS module which allows its easy integration in robotics projects.

    @incollection{lirolem14895,
              volume = {8717},
               month = {September},
              author = {Tomas Krajnik and Joao Santos and Bianca Seemann and Tom Duckett},
              series = {Lecture Notes in Computer Science},
           booktitle = {Advances in Autonomous Robotics Systems},
              editor = {Michael Mistry and Ale Leonardis and Mark Witkowski},
               title = {FROctomap: an efficient spatio-temporal environment representation},
           publisher = {Springer International Publishing},
                year = {2014},
               pages = {281--282},
            keywords = {ARRAY(0x7f78593b5f98)},
                 url = {http://eprints.lincoln.ac.uk/14895/},
            abstract = {We present a novel software tool intended for mobile robot mapping in long-term scenarios. The method allows for efficient volumetric representation of dynamic three-dimensional environments over long periods of time. It is based on a combination of a well-established 3D mapping framework called Octomaps and an idea to model environment dynamics by its frequency spectrum. The proposed method allows not only for efficient representation, but also reliable prediction of the future states of dynamic three-dimensional environments. Our spatio-temporal mapping framework is available as an open-source C++ library and a ROS module which allows its easy integration in robotics projects.}
    }
  • T. Krajnik, J. P. Fentanes, O. M. Mozos, T. Duckett, J. Ekekrantz, and M. Hanheide, “Long-term topological localisation for service robots in dynamic environments using spectral maps,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2014.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting. The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.

    @inproceedings{lirolem14423,
           booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
               month = {September},
               title = {Long-term topological localisation for service robots in dynamic environments using spectral maps},
              author = {Tomas Krajnik and Jaime Pulido Fentanes and Oscar Martinez Mozos and Tom Duckett and Johan Ekekrantz and Marc Hanheide},
           publisher = {IEEE},
                year = {2014},
            keywords = {ARRAY(0x7f78592d1958)},
                 url = {http://eprints.lincoln.ac.uk/14423/},
            abstract = {This paper presents a new approach for topological localisation of service robots in dynamic indoor environments. In contrast to typical localisation approaches that rely mainly on static parts of the environment, our approach makes explicit use of information about changes by learning and modelling the spatio-temporal dynamics of the environment where the robot is acting.  The proposed spatio-temporal world model is able to predict environmental changes in time, allowing the robot to improve its localisation capabilities during long-term operations in populated environments. To investigate the proposed approach, we have enabled a mobile robot to autonomously patrol a populated environment over a period of one week while building the proposed model representation. We demonstrate that the experience learned during one week is applicable for topological localization even after a hiatus of three months by showing that the localization error rate is significantly lower compared to static environment representations.}
    }
  • O. Kroemer, V. H. Hoof, G. Neumann, and J. Peters, “Learning to predict phases of manipulation tasks as hidden states,” in 2014 IEEE International Conference on Robotics and Automation, 2014, pp. 4009-4014.
    [BibTeX] [Abstract] [EPrints]

    Phase transitions in manipulation tasks often occur when contacts between objects are made or broken. A switch of the phase can result in the robot?s actions suddenly influencing different aspects of its environment. Therefore, the boundaries between phases often correspond to constraints or subgoals of the manipulation task. In this paper, we investigate how the phases of manipulation tasks can be learned from data. The task is modeled as an autoregressive hidden Markov model, wherein the hidden phase transitions depend on the observed states. The model is learned from data using the expectation-maximization algorithm. We demonstrate the proposed method on both a pushing task and a pepper mill turning task. The proposed approach was compared to a standard autoregressive hidden Markov model. The experiments show that the learned models can accurately predict the transitions in phases during the manipulation tasks.

    @inproceedings{lirolem25769,
           booktitle = {2014 IEEE International Conference on Robotics and Automation},
               month = {September},
               title = {Learning to predict phases of manipulation tasks as hidden states},
              author = {O. Kroemer and H. Van Hoof and G. Neumann and J. Peters},
                year = {2014},
               pages = {4009--4014},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
            keywords = {ARRAY(0x7f78593b5ff8)},
                 url = {http://eprints.lincoln.ac.uk/25769/},
            abstract = {Phase transitions in manipulation tasks often occur
    when contacts between objects are made or broken. A
    switch of the phase can result in the robot?s actions suddenly
    influencing different aspects of its environment. Therefore, the
    boundaries between phases often correspond to constraints or
    subgoals of the manipulation task.
    In this paper, we investigate how the phases of manipulation
    tasks can be learned from data. The task is modeled as an
    autoregressive hidden Markov model, wherein the hidden phase
    transitions depend on the observed states. The model is learned
    from data using the expectation-maximization algorithm. We
    demonstrate the proposed method on both a pushing task
    and a pepper mill turning task. The proposed approach was
    compared to a standard autoregressive hidden Markov model.
    The experiments show that the learned models can accurately
    predict the transitions in phases during the manipulation tasks.}
    }
  • S. Lemaignan, M. Hanheide, M. Karg, H. Khambhaita, L. Kunze, F. Lier, I. L~A?tkebohle, and G. Milliez, “Simulation and HRI recent perspectives with the MORSE simulator,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 8810, pp. 13-24, 2014.
    [BibTeX] [Abstract] [EPrints]

    Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate. Â\copyright 2014 Springer International Publishing Switzerland.

    @article{lirolem21430,
              volume = {8810},
               month = {October},
              author = {S. Lemaignan and M. Hanheide and M. Karg and H. Khambhaita and L. Kunze and F. Lier and I. L{\~A}?tkebohle and G. Milliez},
                note = {Find out how to access preview-only content
    Chapter
    Simulation, Modeling, and Programming for Autonomous Robots
    Volume 8810 of the series Lecture Notes in Computer Science pp 13-24
    Simulation and HRI Recent Perspectives with the MORSE Simulator
    
    S{\'e}verin Lemaignan, Marc Hanheide, Michael Karg, Harmish Khambhaita, Lars Kunze, Florian Lier, Ingo L{\"u}tkebohle, Gr{\'e}goire Milliez
    Buy chapter
    \$29.95 / ?24.95 / {\pounds}19.95 *
    Buy eBook
    \$89.00 / ?67.82 / {\pounds}56.99*
    * Final gross prices may vary according to local VAT.Get Access
    Abstract
    Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate.
    Simulation, Modeling, and Programming for Autonomous RobotsSimulation, Modeling, and Programming for Autonomous Robots Look 
    Inside
    Chapter Metrics
    Downloads
    367
    Provided by Bookmetrix
    Reference tools
    Export citation
    Add to Papers
    Other actions
    About this Book
    Reprints and Permissions
    Share
    Share this content on Facebook Share this content on Twitter Share this content on LinkedIn
    Supplementary Material (0)
    References (24)
    
    About this Chapter
    Title
    Simulation and HRI Recent Perspectives with the MORSE Simulator
    Book Title
    Simulation, Modeling, and Programming for Autonomous Robots
    Book Subtitle
    4th International Conference, SIMPAR 2014, Bergamo, Italy, October 20-23, 2014. Proceedings},
               title = {Simulation and HRI recent perspectives with the MORSE simulator},
           publisher = {Springer Verlag},
                year = {2014},
             journal = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)},
               pages = {13--24},
            keywords = {ARRAY(0x7f78592e2b88)},
                 url = {http://eprints.lincoln.ac.uk/21430/},
            abstract = {Simulation in robotics is often a love-hate relationship: while simulators do save us a lot of time and effort compared to regular deployment of complex software architectures on complex hardware, simulators are also known to evade many of the real issues that robots need to manage when they enter the real world. Because humans are the paragon of dynamic, unpredictable, complex, real world entities, simulation of human-robot interactions may look condemn to fail, or, in the best case, to be mostly useless. This collective article reports on five independent applications of the MORSE simulator in the field of human-robot interaction: It appears that simulation is already useful, if not essential, to successfully carry out research in the field of HRI, and sometimes in scenarios we do not anticipate. {\^A}{\copyright} 2014 Springer International Publishing Switzerland.}
    }
  • R. Lioutikov, A. Paraschos, J. Peters, and G. Neumann, “Generalizing movements with information-theoretic stochastic optimal control,” Journal of Aerospace Information Systems, vol. 11, iss. 9, pp. 579-595, 2014.
    [BibTeX] [Abstract] [EPrints]

    Stochastic optimal control is typically used to plan a movement for a specific situation. Although most stochastic optimal control methods fail to generalize this movement plan to a new situation without replanning, a stochastic optimal control method is presented that allows reuse of the obtained policy in a new situation, as the policy is more robust to slight deviations from the initial movement plan. To improve the robustness of the policy, we employ information-theoretic policy updates that explicitly operate on trajectory distributions instead of single trajectories. To ensure a stable and smooth policy update, the ?distance? is limited between the trajectory distributions of the old and the new control policies. The introduced bound offers a closed-form solution for the resulting policy and extends results from recent developments in stochastic optimal control. In contrast to many standard stochastic optimal control algorithms, the current approach can directly infer the system dynamics from data points, and hence can also be used for model-based reinforcement learning. This paper represents an extension of the paper by Lioutikov et al. (?Sample-Based Information-Theoretic Stochastic Optimal Control,? Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, 2014, pp. 3896?3902). In addition to revisiting the content, an extensive theoretical comparison is presented of the approach with related work, additional aspects of the implementation are discussed, and further evaluations are introduced.

    @article{lirolem25767,
              volume = {11},
              number = {9},
               month = {September},
              author = {R. Lioutikov and A. Paraschos and J. Peters and G. Neumann},
               title = {Generalizing movements with information-theoretic stochastic optimal control},
           publisher = {American Institute of Aeronautics and Astronautics},
                year = {2014},
             journal = {Journal of Aerospace Information Systems},
               pages = {579--595},
            keywords = {ARRAY(0x7f78592e23a8)},
                 url = {http://eprints.lincoln.ac.uk/25767/},
            abstract = {Stochastic optimal control is typically used to plan a movement for a specific situation. Although most stochastic optimal control methods fail to generalize this movement plan to a new situation without replanning, a stochastic optimal control method is presented that allows reuse of the obtained policy in a new situation, as the policy is more robust to slight deviations from the initial movement plan. To improve the robustness of the policy, we employ information-theoretic policy updates that explicitly operate on trajectory distributions instead of single trajectories. To ensure a stable and smooth policy update, the ?distance? is limited between the trajectory distributions of the old and the new control policies. The introduced bound offers a closed-form solution for the resulting policy and extends results from recent developments in stochastic optimal control. In contrast to many standard stochastic optimal control algorithms, the current approach can directly infer the system dynamics from data points, and hence can also be used for model-based reinforcement learning. This paper represents an extension of the paper by Lioutikov et al. (?Sample-Based Information-Theoretic Stochastic Optimal Control,? Proceedings of 2014 IEEE International Conference on Robotics and Automation (ICRA), IEEE, Piscataway, NJ, 2014, pp. 3896?3902). In addition to revisiting the content, an extensive theoretical comparison is presented of the approach with related work, additional aspects of the implementation are discussed, and further evaluations are introduced.}
    }
  • R. Lioutikov, A. Paraschos, J. Peters, and G. Neumann, “Sample-based information-theoretic stochastic optimal control,” in Proceedings of 2014 IEEE International Conference on Robotics and Automation, 2014, pp. 3896-3902.
    [BibTeX] [Abstract] [EPrints]

    Many Stochastic Optimal Control (SOC) approaches rely on samples to either obtain an estimate of the value function or a linearisation of the underlying system model. However, these approaches typically neglect the fact that the accuracy of the policy update depends on the closeness of the resulting trajectory distribution to these samples. The greedy operator does not consider such closeness constraint to the samples. Hence, the greedy operator can lead to oscillations or even instabilities in the policy updates. Such undesired behaviour is likely to result in an inferior performance of the estimated policy. We reuse inspiration from the reinforcement learning community and relax the greedy operator used in SOC with an information theoretic bound that limits the ?distance? of two subsequent trajectory distributions in a policy update. The introduced bound ensures a smooth and stable policy update. Our method is also well suited for model-based reinforcement learning, where we estimate the system dynamics model from data. As this model is likely to be inaccurate, it might be dangerous to exploit the model greedily. Instead, our bound ensures that we generate new data in the vicinity of the current data, such that we can improve our estimate of the system dynamics model. We show that our approach outperforms several state of the art approaches on challenging simulated robot control tasks.

    @inproceedings{lirolem25771,
           booktitle = {Proceedings of 2014 IEEE International Conference on Robotics and Automation},
               month = {September},
               title = {Sample-based information-theoretic stochastic optimal control},
              author = {R. Lioutikov and A. Paraschos and J. Peters and G. Neumann},
                year = {2014},
               pages = {3896--3902},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
            keywords = {ARRAY(0x7f78593b5638)},
                 url = {http://eprints.lincoln.ac.uk/25771/},
            abstract = {Many Stochastic Optimal Control (SOC) approaches
    rely on samples to either obtain an estimate of the
    value function or a linearisation of the underlying system model.
    However, these approaches typically neglect the fact that the
    accuracy of the policy update depends on the closeness of the
    resulting trajectory distribution to these samples. The greedy
    operator does not consider such closeness constraint to the
    samples. Hence, the greedy operator can lead to oscillations
    or even instabilities in the policy updates. Such undesired
    behaviour is likely to result in an inferior performance of the
    estimated policy. We reuse inspiration from the reinforcement
    learning community and relax the greedy operator used in SOC
    with an information theoretic bound that limits the ?distance? of
    two subsequent trajectory distributions in a policy update. The
    introduced bound ensures a smooth and stable policy update.
    Our method is also well suited for model-based reinforcement
    learning, where we estimate the system dynamics model from
    data. As this model is likely to be inaccurate, it might be
    dangerous to exploit the model greedily. Instead, our bound
    ensures that we generate new data in the vicinity of the current
    data, such that we can improve our estimate of the system
    dynamics model. We show that our approach outperforms
    several state of the art approaches on challenging simulated
    robot control tasks.}
    }
  • H. Liu and S. Yue, “An efficient method to structural static reanalysis with deleting support constraints,” Structural Engineering and Mechanics, vol. 52, iss. 6, pp. 1121-1134, 2014.
    [BibTeX] [Abstract] [EPrints]

    Structural design is usually an optimization process. Numerous parameters such as the member shapes and sizes, the elasticity modulus of material, the locations of nodes and the support constraints can be selected as design variables. These variables are progressively revised in order to obtain a satisfactory structure. Each modification requires a fresh analysis for the displacements and stresses, and reanalysis can be employed to reduce the computational cost. This paper is focused on static reanalysis problem with modification of deleting some supports. An efficient reanalysis method is proposed. The method makes full use of the initial information and preserves the ease of implementation. Numerical examples show that the calculated results of the proposed method are the identical as those of the direct analysis, while the computational time is remarkably reduced.

    @article{lirolem16505,
              volume = {52},
              number = {6},
               month = {December},
              author = {H. Liu and Shigang Yue},
               title = {An efficient method to structural static reanalysis with deleting support constraints},
           publisher = {Techno Press},
                year = {2014},
             journal = {Structural Engineering and Mechanics},
               pages = {1121--1134},
            keywords = {ARRAY(0x7f7859456848)},
                 url = {http://eprints.lincoln.ac.uk/16505/},
            abstract = {Structural design is usually an optimization process. Numerous parameters such as the member shapes and sizes, the elasticity modulus of material, the locations of nodes and the support constraints can be selected as design variables. These variables are progressively revised in order to obtain a satisfactory structure. Each modification requires a fresh analysis for the displacements and stresses, and reanalysis can be employed to reduce the computational cost. This paper is focused on static reanalysis problem with modification of deleting some supports. An efficient reanalysis method is proposed. The method makes full use of the initial information and preserves the ease of implementation. Numerical examples show that the calculated results of the proposed method are the identical as those of the direct analysis, while the computational time is remarkably reduced.}
    }
  • D. Liu and S. Yue, “Spiking neural network for visual pattern recognition,” in International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014, 2014, pp. 1-5.
    [BibTeX] [Abstract] [EPrints]

    Most of visual pattern recognition algorithms try to emulate the mechanism of visual pathway within the human brain. Regarding of classic face recognition task, by using the spatiotemporal information extracted from Spiking neural network (SNN), batch learning rule and on-line learning rule stand out from their competitors. However, the former one simply considers the average pattern within the class, and the latter one just relies on the nearest relevant single pattern. In this paper, a novel learning rule and its SNN framework has been proposed. It considers all relevant patterns in the local domain around the undetermined sample rather than just nearest relevant single pattern. Experimental results show the proposed learning rule and its SNN framework obtains satisfactory testing results under the ORL face database.

    @inproceedings{lirolem16638,
           booktitle = {International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
               title = {Spiking neural network for visual pattern recognition},
              author = {Daqi Liu and Shigang Yue},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
                year = {2014},
               pages = {1--5},
             journal = {Processing of 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
            keywords = {ARRAY(0x7f7858f95158)},
                 url = {http://eprints.lincoln.ac.uk/16638/},
            abstract = {Most of visual pattern recognition algorithms try to emulate the mechanism of visual pathway within the human brain. Regarding of classic face recognition task, by using the spatiotemporal information extracted from Spiking neural network (SNN), batch learning rule and on-line learning rule stand out from their competitors. However, the former one simply considers the average pattern within the class, and the latter one just relies on the nearest relevant single pattern. In this paper, a novel learning rule and its SNN framework has been proposed. It considers all relevant patterns in the local domain around the undetermined sample rather than just nearest relevant single pattern. Experimental results show the proposed learning rule and its SNN framework obtains satisfactory testing results under the ORL face database.}
    }
  • K. S. Luck, G. Neumann, E. Berger, J. Peters, and H. B. Amor, “Latent space policy search for robotics,” in IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), 2014, pp. 1434-1440.
    [BibTeX] [Abstract] [EPrints]

    Learning motor skills for robots is a hard task. In particular, a high number of degrees-of-freedom in the robot can pose serious challenges to existing reinforcement learning methods, since it leads to a highdimensional search space. However, complex robots are often intrinsically redundant systems and, therefore, can be controlled using a latent manifold of much smaller dimensionality. In this paper, we present a novel policy search method that performs efficient reinforcement learning by uncovering the low-dimensional latent space of actuator redundancies. In contrast to previous attempts at combining reinforcement learning and dimensionality reduction, our approach does not perform dimensionality reduction as a preprocessing step but naturally combines it with policy search. Our evaluations show that the new approach outperforms existing algorithms for learning motor skills with high-dimensional robots.

    @inproceedings{lirolem25772,
           booktitle = {IEEE/RSJ Conference on Intelligent Robots and Systems (IROS)},
               month = {September},
               title = {Latent space policy search for robotics},
              author = {K. S. Luck and G. Neumann and E. Berger and J. Peters and H. B. Amor},
                year = {2014},
               pages = {1434--1440},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
            keywords = {ARRAY(0x7f7859454590)},
                 url = {http://eprints.lincoln.ac.uk/25772/},
            abstract = {Learning motor skills for robots is a hard
    task. In particular, a high number of degrees-of-freedom
    in the robot can pose serious challenges to existing reinforcement
    learning methods, since it leads to a highdimensional
    search space. However, complex robots are
    often intrinsically redundant systems and, therefore, can
    be controlled using a latent manifold of much smaller
    dimensionality. In this paper, we present a novel policy
    search method that performs efficient reinforcement learning
    by uncovering the low-dimensional latent space of
    actuator redundancies. In contrast to previous attempts
    at combining reinforcement learning and dimensionality
    reduction, our approach does not perform dimensionality
    reduction as a preprocessing step but naturally combines
    it with policy search. Our evaluations show that the new
    approach outperforms existing algorithms for learning
    motor skills with high-dimensional robots.}
    }
  • G. Maeda, M. Ewerton, R. Lioutikov, B. H. Amor, J. Peters, and G. Neumann, “Learning interaction for collaborative tasks with probabilistic movement primitives,” in 14th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2014, pp. 527-534.
    [BibTeX] [Abstract] [EPrints]

    This paper proposes a probabilistic framework based on movement primitives for robots that work in collaboration with a human coworker. Since the human coworker can execute a variety of unforeseen tasks a requirement of our system is that the robot assistant must be able to adapt and learn new skills on-demand, without the need of an expert programmer. Thus, this paper leverages on the framework of imitation learning and its application to human-robot interaction using the concept of Interaction Primitives (IPs). We introduce the use of Probabilistic Movement Primitives (ProMPs) to devise an interaction method that both recognizes the action of a human and generates the appropriate movement primitive of the robot assistant. We evaluate our method on experiments using a lightweight arm interacting with a human partner and also using motion capture trajectories of two humans assembling a box. The advantages of ProMPs in relation to the original formulation for interaction are exposed and compared.

    @inproceedings{lirolem25764,
              volume = {2015-F},
               month = {November},
              author = {G. Maeda and M. Ewerton and R. Lioutikov and H. Ben Amor and J. Peters and G. Neumann},
           booktitle = {14th IEEE-RAS International Conference on Humanoid Robots (Humanoids)},
               title = {Learning interaction for collaborative tasks with probabilistic movement primitives},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {527--534},
                year = {2014},
            keywords = {ARRAY(0x7f78593b5e18)},
                 url = {http://eprints.lincoln.ac.uk/25764/},
            abstract = {This paper proposes a probabilistic framework
    based on movement primitives for robots that work in collaboration
    with a human coworker. Since the human coworker
    can execute a variety of unforeseen tasks a requirement of our
    system is that the robot assistant must be able to adapt and
    learn new skills on-demand, without the need of an expert
    programmer. Thus, this paper leverages on the framework
    of imitation learning and its application to human-robot interaction
    using the concept of Interaction Primitives (IPs).
    We introduce the use of Probabilistic Movement Primitives
    (ProMPs) to devise an interaction method that both recognizes
    the action of a human and generates the appropriate movement
    primitive of the robot assistant. We evaluate our method
    on experiments using a lightweight arm interacting with a
    human partner and also using motion capture trajectories of
    two humans assembling a box. The advantages of ProMPs in
    relation to the original formulation for interaction are exposed
    and compared.}
    }
  • F. Moreno, G. Cielniak, and T. Duckett, “Evaluation of laser range-finder mapping for agricultural spraying vehicles,” in Towards autonomous robotic systems, A. Natraj, S. Cameron, C. Melhuish, and M. Witkowski, Eds., Springer Berlin Heidelberg, 2014, vol. 8069, pp. 210-221.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer?s cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.

    @incollection{lirolem19647,
              volume = {8069},
               month = {June},
              author = {Francisco-Angel Moreno and Grzegorz Cielniak and Tom Duckett},
              series = {Lecture Notes in Computer Science},
                note = {14th Annual Conference, TAROS 2013, Oxford, UK, August 28--30, 2013, Revised Selected Papers},
           booktitle = {Towards autonomous robotic systems},
              editor = {Ashutosh Natraj and Stephen Cameron and Chris Melhuish and Mark Witkowski},
               title = {Evaluation of laser range-finder mapping for agricultural spraying vehicles},
           publisher = {Springer Berlin Heidelberg},
                year = {2014},
               pages = {210--221},
            keywords = {ARRAY(0x7f78593a9a40)},
                 url = {http://eprints.lincoln.ac.uk/19647/},
            abstract = {In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer?s cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.}
    }
  • G. Neumann, C. Daniel, A. Paraschos, A. Kupcsik, and J. Peters, “Learning modular policies for robotics,” Frontiers in Computational Neuroscience, vol. 8, iss. JUN, 2014.
    [BibTeX] [Abstract] [EPrints]

    A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots.

    @article{lirolem25765,
              volume = {8},
              number = {JUN},
               month = {June},
              author = {G. Neumann and C. Daniel and A. Paraschos and A. Kupcsik and J. Peters},
               title = {Learning modular policies for robotics},
           publisher = {Frontiers Media},
             journal = {Frontiers in Computational Neuroscience},
                year = {2014},
            keywords = {ARRAY(0x7f78593b60e8)},
                 url = {http://eprints.lincoln.ac.uk/25765/},
            abstract = {A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots.}
    }
  • E. Rueckert, M. Mindt, J. Peters, and G. Neumann, “Robust policy updates for stochastic optimal control,” in Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on, 2014, pp. 388-393.
    [BibTeX] [Abstract] [EPrints]

    For controlling high-dimensional robots, most stochastic optimal control algorithms use approximations of the system dynamics and of the cost function (e.g., using linearizations and Taylor expansions). These approximations are typically only locally correct, which might cause instabilities in the greedy policy updates, lead to oscillations or the algorithms diverge. To overcome these drawbacks, we add a regularization term to the cost function that punishes large policy update steps in the trajectory optimization procedure. We applied this concept to the Approximate Inference Control method (AICO), where the resulting algorithm guarantees convergence for uninformative initial solutions without complex hand-tuning of learning rates. We evaluated our new algorithm on two simulated robotic platforms. A robot arm with five joints was used for reaching multiple targets while keeping the roll angle constant. On the humanoid robot Nao, we show how complex skills like reaching and balancing can be inferred from desired center of gravity or end effector coordinates.

    @inproceedings{lirolem25754,
              volume = {2015-F},
               month = {November},
              author = {E. Rueckert and M. Mindt and J. Peters and G. Neumann},
           booktitle = {Humanoid Robots (Humanoids), 2014 14th IEEE-RAS International Conference on},
               title = {Robust policy updates for stochastic optimal control},
             journal = {IEEE-RAS International Conference on Humanoid Robots},
               pages = {388--393},
                year = {2014},
            keywords = {ARRAY(0x7f78593b55c0)},
                 url = {http://eprints.lincoln.ac.uk/25754/},
            abstract = {For controlling high-dimensional robots, most stochastic optimal control algorithms use approximations of the system dynamics and of the cost function (e.g., using linearizations and Taylor expansions). These approximations are typically only locally correct, which might cause instabilities in the greedy policy updates, lead to oscillations or the algorithms diverge. To overcome these drawbacks, we add a regularization term to the cost function that punishes large policy update steps in the trajectory optimization procedure. We applied this concept to the Approximate Inference Control method (AICO), where the resulting algorithm guarantees convergence for uninformative initial solutions without complex hand-tuning of learning rates. We evaluated our new algorithm on two simulated robotic platforms. A robot arm with five joints was used for reaching multiple targets while keeping the roll angle constant. On the humanoid robot Nao, we show how complex skills like reaching and balancing can be inferred from desired center of gravity or end effector coordinates.}
    }
  • L. Shi, C. Zhang, and S. Yue, “Vector control IC for permanent magnet synchronous motor,” in 2014 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC), 2014.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a full-digital vector control integrated circuit(IC) for permanent magnet synchronous motor (PMSM) with considering hardware structure. We adopted top-down and modular partitioning logic optimization design. Design specification of space vector pulse width modulation (SVPWM) unit, vector coordinate transformation are illustrated. All of the modules were implemented with pure hardware and designed with Verilog hardware description language (HDL). Moreover, the proposed design was verified by Simulink-Matlab and field programmable gate array (FPGA). \copyright 2014 IEEE.

    @inproceedings{lirolem17531,
               month = {June},
              author = {Licheng Shi and Chun Zhang and Shigang Yue},
                note = {Conference Code:111593},
           booktitle = {2014 IEEE International Conference on Electron Devices and Solid-State Circuits (EDSSC)},
               title = {Vector control IC for permanent magnet synchronous motor},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
             journal = {2014 IEEE International Conference on Electron Devices and Solid-State Circuits, EDSSC 2014},
                year = {2014},
            keywords = {ARRAY(0x7f78593b5ea8)},
                 url = {http://eprints.lincoln.ac.uk/17531/},
            abstract = {This paper presents a full-digital vector control integrated circuit(IC) for permanent magnet synchronous motor (PMSM) with considering hardware structure. We adopted top-down and modular partitioning logic optimization design. Design specification of space vector pulse width modulation (SVPWM) unit, vector coordinate transformation are illustrated. All of the modules were implemented with pure hardware and designed with Verilog hardware description language (HDL). Moreover, the proposed design was verified by Simulink-Matlab and field programmable gate array (FPGA). {\copyright} 2014 IEEE.}
    }
  • T. Stone, M. Mangan, P. Ardin, and B. Webb, “Sky segmentation with ultraviolet images can be used for navigation,” in 2014 Robotics: Science and Systems Conference, 2014.
    [BibTeX] [Abstract] [EPrints]

    Inspired by ant navigation, we explore a method for sky segmentation using ultraviolet (UV) light. A standard camera is adapted to allow collection of outdoor images containing light in the visible range, in UV only and in green only. Automatic segmentation of the sky region using UV only is significantly more accurate and far more consistent than visible wavelengths over a wide range of locations, times and weather conditions, and can be accomplished with a very low complexity algorithm. We apply this method to obtain compact binary (sky vs non-sky) images from panoramic UV images taken along a 2km route in an urban environment. Using either sequence SLAM or a visual compass on these images produces reliable localisation and orientation on a subsequent traversal of the route under different weather conditions.

    @inproceedings{lirolem24748,
           booktitle = {2014 Robotics: Science and Systems Conference},
               month = {July},
               title = {Sky segmentation with ultraviolet images can be used for navigation},
              author = {Thomas Stone and Michael Mangan and Paul Ardin and Barbara Webb},
           publisher = {Robotics: Science and Systems},
                year = {2014},
             journal = {Robotics: Science and Systems},
            keywords = {ARRAY(0x7f7859415448)},
                 url = {http://eprints.lincoln.ac.uk/24748/},
            abstract = {Inspired by ant navigation, we explore a method for sky segmentation using ultraviolet (UV) light. A standard camera is adapted to allow collection of outdoor images containing light in the visible range, in UV only and in green only. Automatic segmentation of the sky region using UV only is significantly more accurate and far more consistent than visible wavelengths over a wide range of locations, times and weather conditions, and can be accomplished with a very low complexity algorithm. We apply this method to obtain compact binary (sky vs non-sky) images from panoramic UV images taken along a 2km route in an urban environment. Using either sequence SLAM or a visual compass on these images produces reliable localisation and orientation on a subsequent traversal of the route under different weather conditions.}
    }
  • Y. Tang, J. Peng, and S. Yue, “Cyclic and simultaneous iterative methods to matrix equations of the form AiX Bi = Fi,” Numerical Algorithms, vol. 66, iss. 2, pp. 379-397, 2014.
    [BibTeX] [Abstract] [EPrints]

    This paper deals with a general type of linear matrix equation problem. It presents new iterative algorithms to solve the matrix equations of the form AiX Bi = Fi. These algorithms are based on the incremental subgradient and the parallel subgradient methods. The convergence region of these algorithms are larger than other existing iterative algorithms. Finally, some experimental results are presented to show the efficiency of the proposed algorithms. Â\copyright 2013 Springer Science+Business Media New York.

    @article{lirolem11574,
              volume = {66},
              number = {2},
               month = {June},
              author = {Yuchao Tang and Jigen Peng and Shigang Yue},
               title = {Cyclic and simultaneous iterative methods to matrix equations of the form AiX Bi = Fi},
           publisher = {Springer},
                year = {2014},
             journal = {Numerical Algorithms},
               pages = {379--397},
            keywords = {ARRAY(0x7f78593b5f20)},
                 url = {http://eprints.lincoln.ac.uk/11574/},
            abstract = {This paper deals with a general type of linear matrix equation problem. It presents new iterative algorithms to solve the matrix equations of the form AiX Bi = Fi. These algorithms are based on the incremental subgradient and the parallel subgradient methods. The convergence region of these algorithms are larger than other existing iterative algorithms. Finally, some experimental results are presented to show the efficiency of the proposed algorithms. {\^A}{\copyright} 2013 Springer Science+Business Media New York.}
    }
  • P. Urcola, T. Duckett, and G. Cielniak, “On-line trajectory planning for autonomous spraying vehicles,” in International Workshop on Recent Advances in Agricultural Robotics, 2014.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a new application of on-line trajectory planning for autonomous sprayers. The current generation of these vehicles use automatic controllers to maintain the height of the spraying booms above the crop. However, such systems are typically based on ultrasonic sensors mounted directly on the booms, which limits the response of the controller to changes in the terrain, resulting in a suboptimal spraying process. To overcome these limitations, we propose to use 3D maps of the terrain ahead of the spraying booms based on laser range-fi?nder measurements combined with GPS-based localisation. Four different boom trajectory planning solutions which utilise the 3D maps are considered and their accuracy and real-time suitability is evaluated based on data collected from ?field tests. The point optimisation and interpolation technique presents a practical solution demonstrating satisfactory performance under real-time constraints.

    @inproceedings{lirolem14603,
           booktitle = {International Workshop on Recent Advances in Agricultural Robotics},
               month = {July},
               title = {On-line trajectory planning for autonomous spraying vehicles},
              author = {Pablo Urcola and Tom Duckett and Grzegorz Cielniak},
                year = {2014},
            keywords = {ARRAY(0x7f78593b5608)},
                 url = {http://eprints.lincoln.ac.uk/14603/},
            abstract = {In this paper, we present a new application of on-line trajectory planning for autonomous sprayers. The current generation of these vehicles use automatic controllers to maintain the height of the spraying booms above the crop. However, such systems are typically based on ultrasonic sensors mounted directly on the booms, which limits the
    response of the controller to changes in the terrain, resulting in a suboptimal spraying process. To overcome these limitations, we propose to use 3D maps of the terrain ahead of the spraying booms based on laser range-fi?nder measurements combined with GPS-based localisation. Four different boom trajectory planning solutions which utilise the 3D maps are considered and their accuracy and real-time suitability is evaluated based on data collected from ?field tests. The point optimisation and interpolation technique presents a practical solution demonstrating satisfactory performance under real-time constraints.}
    }
  • J. Xu and S. Yue, “Mimicking visual searching with integrated top down cues and low-level features,” Neurocomputing, vol. 133, pp. 1-17, 2014.
    [BibTeX] [Abstract] [EPrints]

    Visual searching is a perception task involved with visual attention, attention shift and active scan of the visual environment for a particular object or feature. The key idea of our paper is to mimic the human visual searching under the static and dynamic scenes. To build up an artificial vision system that performs the visual searching could be helpful to medical and psychological application development to human machine interaction. Recent state-of-the-art researches focus on the bottom-up and top-down saliency maps. Saliency maps indicate that the saliency likelihood of each pixel, however, understanding the visual searching process can help an artificial vision system exam details in a way similar to human and they will be good for future robots or machine vision systems which is a deeper digest than the saliency map. This paper proposed a computational model trying to mimic human visual searching process and we emphasis the motion cues on the visual processing and searching. Our model analysis the attention shifts by fusing the top-down bias and bottom-up cues. This model also takes account the motion factor into the visual searching processing. The proposed model involves five modules: the pre-learning process; top-down biasing; bottom-up mechanism; multi-layer neural network and attention shifts. Experiment evaluation results via benchmark databases and real-time video showed the model demonstrated high robustness and real-time ability under complex dynamic scenes.

    @article{lirolem13453,
              volume = {133},
               month = {June},
              author = {Jiawei Xu and Shigang Yue},
               title = {Mimicking visual searching with integrated top down cues and low-level features},
           publisher = {Elsevier},
             journal = {Neurocomputing},
               pages = {1--17},
                year = {2014},
            keywords = {ARRAY(0x7f78593af040)},
                 url = {http://eprints.lincoln.ac.uk/13453/},
            abstract = {Visual searching is a perception task involved with visual attention, attention shift and active scan of the visual environment for a particular object or feature. The key idea of our paper is to mimic the human visual searching under the static and dynamic scenes. To build up an artificial vision system that performs the visual searching could be helpful to medical and psychological application development to human machine interaction. Recent state-of-the-art researches focus on the bottom-up and top-down saliency maps. Saliency maps indicate that the saliency likelihood of each pixel, however, understanding the visual searching process can help an artificial vision system exam details in a way similar to human and they will be good for future robots or machine vision systems which is a deeper digest than the saliency map. This paper proposed a computational model trying to mimic human visual searching process and we emphasis the motion cues on the visual processing and searching. Our model analysis the attention shifts by fusing the top-down bias and bottom-up cues. This model also takes account the motion factor into the visual searching processing. The proposed model involves five modules: the pre-learning process; top-down biasing; bottom-up mechanism; multi-layer neural network and attention shifts. Experiment evaluation results via benchmark databases and real-time video showed the model demonstrated high robustness and real-time ability under complex dynamic scenes.}
    }
  • J. Xu, R. Wang, and S. Yue, “Bio-inspired classifier for road extraction from remote sensing imagery,” Journal of Applied Remote Sensing, vol. 8, iss. 1, p. 83577, 2014.
    [BibTeX] [Abstract] [EPrints]

    An adaptive approach for road extraction inspired by the mechanism of primary visual cortex (V1) is proposed. The motivation is originated by the characteristics in the receptive field from V1. It has been proved that human or primate visual systems can distinguish useful cues from real scenes effortlessly while traditional computer vision techniques cannot accomplish this task easily. This idea motivates us to design a bio-inspired model for road extraction from remote sensing imagery. The proposed approach is an improved support vector machine (SVM) based on the pooling of feature vectors, using an improved Gaussian radial basis function (RBF) kernel with tuning on synaptic gains. The synaptic gains comprise the feature vectors through an iterative optimization process representing the strength and width of Gaussian RBF kernel. The synaptic gains integrate the excitation and inhibition stimuli based on internal connections from V1. The summation of synaptic gains contributes to pooling of feature vectors. The experimental results verify the correlation between the synaptic gain and classification rules, and then show better performance in comparison with hidden Markov model, SVM, and fuzzy classification approaches. Our contribution is an automatic approach to road extraction without prelabeling and postprocessing work. Another apparent advantage is that our method is robust for images taken even under complex weather conditions such as snowy and foggy weather. Â\copyright 2014 SPIE.

    @article{lirolem14764,
              volume = {8},
              number = {1},
               month = {August},
              author = {Jiawei Xu and Ruisheng Wang and Shigang Yue},
               title = {Bio-inspired classifier for road extraction from remote sensing imagery},
           publisher = {Society of Photo-optical Instrumentation Engineers (SPIE)},
                year = {2014},
             journal = {Journal of Applied Remote Sensing},
               pages = {083577},
            keywords = {ARRAY(0x7f78592e4360)},
                 url = {http://eprints.lincoln.ac.uk/14764/},
            abstract = {An adaptive approach for road extraction inspired by the mechanism of primary visual cortex (V1) is proposed. The motivation is originated by the characteristics in the receptive field from V1. It has been proved that human or primate visual systems can distinguish useful cues from real scenes effortlessly while traditional computer vision techniques cannot accomplish this task easily. This idea motivates us to design a bio-inspired model for road extraction from remote sensing imagery. The proposed approach is an improved support vector machine (SVM) based on the pooling of feature vectors, using an improved Gaussian radial basis function (RBF) kernel with tuning on synaptic gains. The synaptic gains comprise the feature vectors through an iterative optimization process representing the strength and width of Gaussian RBF kernel. The synaptic gains integrate the excitation and inhibition stimuli based on internal connections from V1. The summation of synaptic gains contributes to pooling of feature vectors. The experimental results verify the correlation between the synaptic gain and classification rules, and then show better performance in comparison with hidden Markov model, SVM, and fuzzy classification approaches. Our contribution is an automatic approach to road extraction without prelabeling and postprocessing work. Another apparent advantage is that our method is robust for images taken even under complex weather conditions such as snowy and foggy weather. {\^A}{\copyright} 2014 SPIE.}
    }
  • S. Yue, K. Harmer, K. Guo, K. Adams, and A. Hunter, “Automatic blush detection in ?concealed information? test using visual stimuli,” International Journal of Data Mining, Modelling and Management, vol. 6, iss. 2, pp. 187-201, 2014.
    [BibTeX] [Abstract] [EPrints]

    Blushing has been identified as an indicator of deception, shame, anxiety and embarrassment. Although normally associated with the skin coloration of the face, a blush response also affects skin surface temperature. In this paper, an approach to detect a blush response automatically is presented using the Argus P7225 thermal camera from e2v. The algorithm was tested on a sample population of 51 subjects, while using visual stimuli to elicit a response, and achieved recognition rates of \texttt\char12677\% TPR and \texttt\char12660\% TNR, indicating a thermal image sensor is the prospective device to pick up subtle temperature change synchronised with stimuli.

    @article{lirolem14660,
              volume = {6},
              number = {2},
               month = {June},
              author = {Shigang Yue and Karl Harmer and Kun Guo and Karen Adams and Andrew Hunter},
               title = {Automatic blush detection in ?concealed information? test using visual stimuli},
           publisher = {Inderscience},
                year = {2014},
             journal = {International Journal of Data Mining, Modelling and Management},
               pages = {187--201},
            keywords = {ARRAY(0x7f785945f248)},
                 url = {http://eprints.lincoln.ac.uk/14660/},
            abstract = {Blushing has been identified as an indicator of deception, shame, anxiety and embarrassment. Although normally associated with the skin coloration of the face, a blush response also affects skin surface temperature. In this paper, an approach to detect a blush response automatically is presented using the Argus P7225 thermal camera from e2v. The algorithm was tested on a sample population of 51 subjects, while using visual stimuli to elicit a response,  and achieved recognition rates of {\texttt{\char126}}77\% TPR and {\texttt{\char126}}60\% TNR, indicating a thermal image sensor is the prospective device to pick up subtle temperature change synchronised with stimuli.}
    }
  • G. Zahi and S. Yue, “Reducing motion blurring associated with temporal summation in low light scenes for image quality enhancement,” in International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014, 2014, pp. 1-5.
    [BibTeX] [Abstract] [EPrints]

    In order to see under low light conditions nocturnal insects rely on neural strategies based on combinations of spatial and temporal summations. Though these summation techniques when modelled are effective in improving the quality of low light images, using the temporal summation in scenes where image velocity is high only come at a cost of motion blurring in the output scenes. Most recent research has been towards reducing motion blurring in scenes where motion is caused by moving objects rather than effectively reducing motion blurring in scenes where motion is caused by moving cameras. This makes it impossible to implement the night vision algorithm in moving robots or cars that operate under low light conditions. In this paper we present a generic new method that can replace the normal temporal summation in scenes where motion is detected. The proposed method is both suitable for motion caused by moving objects as well as moving cameras. The effectiveness of this new generic method is shown with relevant supporting experiments.

    @inproceedings{lirolem16637,
           booktitle = {International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
               title = {Reducing motion blurring associated with temporal summation in low light scenes for image quality enhancement},
              author = {Gabriel Zahi and Shigang Yue},
           publisher = {Institute of Electrical and Electronics Engineers Inc.},
                year = {2014},
               pages = {1--5},
             journal = {Processing of 2014 International Conference on Multisensor Fusion and Information Integration for Intelligent Systems, MFI 2014},
            keywords = {ARRAY(0x7f7859102e80)},
                 url = {http://eprints.lincoln.ac.uk/16637/},
            abstract = {In order to see under low light conditions nocturnal insects rely on neural strategies based on combinations of spatial and temporal summations. Though these summation techniques when modelled are effective in improving the quality of low light images, using the temporal summation in scenes where image velocity is high only come at a cost of motion blurring in the output scenes. Most recent research has been towards reducing motion blurring in scenes where motion is caused by moving objects rather than effectively reducing motion blurring in scenes where motion is caused by moving cameras. This makes it impossible to implement the night vision algorithm in moving robots or cars that operate under low light conditions. In this paper we present a generic new method that can replace the normal temporal summation in scenes where motion is detected. The proposed method is both suitable for motion caused by moving objects as well as moving cameras. The effectiveness of this new generic method is shown with relevant supporting experiments.}
    }
  • Z. Zhang, S. Yue, M. Liao, and F. Long, “Danger theory based artificial immune system solving dynamic constrained single-objective optimization,” Soft Computing, vol. 18, iss. 1, pp. 185-206, 2014.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we propose an artificial immune system (AIS) based on the danger theory in immunology for solving dynamic nonlinear constrained single-objective optimization problems with time-dependent design spaces. Such proposed AIS executes orderly three modules-danger detection, immune evolution and memory update. The first module identifies whether there are changes in the optimization environment and decides the environmental level, which helps for creating the initial population in the environment and promoting the process of solution search. The second module runs a loop of optimization, in which three sub-populations each with a dynamic size seek simultaneously the location of the optimal solution along different directions through co-evolution. The last module stores and updates the memory cells which help the first module decide the environmental level. This optimization system is an on-line and adaptive one with the characteristics of simplicity, modularization and co-evolution. The numerical experiments and the results acquired by the nonparametric statistic procedures, based on 22 benchmark problems and an engineering problem, show that the proposed approach performs globally well over the compared algorithms and is of potential use for many kinds of dynamic optimization problems. Â\copyright 2013 Springer-Verlag Berlin Heidelberg.

    @article{lirolem11410,
              volume = {18},
              number = {1},
               month = {January},
              author = {Zhuhong Zhang and Shigang Yue and Min Liao and Fei Long},
               title = {Danger theory based artificial immune system solving dynamic constrained single-objective optimization},
           publisher = {Springer Verlag (Germany)},
                year = {2014},
             journal = {Soft Computing},
               pages = {185--206},
            keywords = {ARRAY(0x7f7858fa0028)},
                 url = {http://eprints.lincoln.ac.uk/11410/},
            abstract = {In this paper, we propose an artificial immune system (AIS) based on the danger theory in immunology for solving dynamic nonlinear constrained single-objective optimization problems with time-dependent design spaces. Such proposed AIS executes orderly three modules-danger detection, immune evolution and memory update. The first module identifies whether there are changes in the optimization environment and decides the environmental level, which helps for creating the initial population in the environment and promoting the process of solution search. The second module runs a loop of optimization, in which three sub-populations each with a dynamic size seek simultaneously the location of the optimal solution along different directions through co-evolution. The last module stores and updates the memory cells which help the first module decide the environmental level. This optimization system is an on-line and adaptive one with the characteristics of simplicity, modularization and co-evolution. The numerical experiments and the results acquired by the nonparametric statistic procedures, based on 22 benchmark problems and an engineering problem, show that the proposed approach performs globally well over the compared algorithms and is of potential use for many kinds of dynamic optimization problems. {\^A}{\copyright} 2013 Springer-Verlag Berlin Heidelberg.}
    }

2013

  • F. Arvin and M. Bekravi, “Encoderless position estimation and error correction techniques for miniature mobile robots,” Turkish Journal of Electrical Engineering & Computer Sciences, vol. 21, iss. 6, pp. 1631-1645, 2013.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an encoderless position estimation technique for miniature-sized mobile robots. Odometry techniques, which are based on the hardware components, are commonly used for calculating the geometric location of mobile robots. Therefore, the robot must be equipped with an appropriate sensor to measure the motion. However, due to the hardware limitations of some robots, employing extra hardware is impossible. On the other hand, in swarm robotic research, which uses a large number of mobile robots, equipping the robots with motion sensors might be costly. In this study, the trajectory of the robot is divided into several small displacements over short spans of time. Therefore, the position of the robot is calculated within a short period, using the speed equations of the robot’s wheel. In addition, an error correction function is proposed that estimates the errors of the motion using a current monitoring technique. The experiments illustrate the feasibility of the proposed position estimation and error correction techniques to be used in miniature-sized mobile robots without requiring an additional sensor.

    @article{lirolem12078,
              volume = {21},
              number = {6},
               month = {October},
              author = {Farshad Arvin and Masoud Bekravi},
               title = {Encoderless position estimation and error correction techniques for miniature mobile robots},
           publisher = {Scientific and Technical Research Council of Turkey},
                year = {2013},
             journal = {Turkish Journal of Electrical Engineering \& Computer Sciences},
               pages = {1631--1645},
            keywords = {ARRAY(0x7f785943a2d0)},
                 url = {http://eprints.lincoln.ac.uk/12078/},
            abstract = { This paper presents an encoderless position estimation technique for miniature-sized mobile robots. Odometry techniques, which are based on the hardware components, are commonly used for calculating the geometric location of mobile robots. Therefore, the robot must be equipped with an appropriate sensor to measure the motion. However, due to the hardware limitations of some robots, employing extra hardware is impossible. On the other hand, in swarm robotic research, which uses a large number of mobile robots, equipping the robots with motion sensors might be costly. In this study, the trajectory of the robot is divided into several small displacements over short spans of time. Therefore, the position of the robot is calculated within a short period, using the speed equations of the robot's wheel. In addition, an error correction function is proposed that estimates the errors of the motion using a current monitoring technique. The experiments illustrate the feasibility of the proposed position estimation and error correction techniques to be used in miniature-sized mobile robots without requiring an additional sensor. }
    }
  • P. E. Baxter, J. de Greeff, and T. Belpaeme, “Cognitive architecture for human?robot interaction: towards behavioural alignment,” Biologically Inspired Cognitive Architectures, vol. 6, pp. 30-39, 2013.
    [BibTeX] [Abstract] [EPrints]

    Abstract With increasingly competent robotic systems desired and required for social human?robot interaction comes the necessity for more complex means of control. Cognitive architectures (specifically the perspective where principles of structure and function are sought to account for multiple cognitive competencies) have only relatively recently been considered for applica- tion to this domain. In this paper, we describe one such set of architectural principles ? acti- vation dynamics over a developmental distributed associative substrate ? and show how this enables an account of a fundamental competence for social cognition: multi-modal behavioural alignment. Data from real human?robot interactions is modelled using a computational system based on this set of principles to demonstrate how this competence can therefore be consid- ered as embedded in wider cognitive processing. It is shown that the proposed system can model the behavioural characteristics of human subjects. While this study is a simulation using real interaction data, the results obtained validate the application of the proposed approach to this issue.

    @article{lirolem23076,
              volume = {6},
               month = {October},
              author = {Paul E. Baxter and Joachim de Greeff and Tony Belpaeme},
               title = {Cognitive architecture for human?robot interaction: towards behavioural alignment},
           publisher = {Elsevier B.V.},
             journal = {Biologically Inspired Cognitive Architectures},
               pages = {30--39},
                year = {2013},
            keywords = {ARRAY(0x7f7859111f28)},
                 url = {http://eprints.lincoln.ac.uk/23076/},
            abstract = {Abstract With increasingly competent robotic systems desired and required for social human?robot interaction comes the necessity for more complex means of control. Cognitive architectures (specifically the perspective where principles of structure and function are sought to account for multiple cognitive competencies) have only relatively recently been considered for applica- tion to this domain. In this paper, we describe one such set of architectural principles ? acti- vation dynamics over a developmental distributed associative substrate ? and show how this enables an account of a fundamental competence for social cognition: multi-modal behavioural alignment. Data from real human?robot interactions is modelled using a computational system based on this set of principles to demonstrate how this competence can therefore be consid- ered as embedded in wider cognitive processing. It is shown that the proposed system can model the behavioural characteristics of human subjects. While this study is a simulation using real interaction data, the results obtained validate the application of the proposed approach to this issue.}
    }
  • P. Baxter, J. D. Greeff, R. Wood, and T. Belpaeme, “Modelling concept prototype competencies using a developmental memory model,” Paladyn, Journal of Behavioral Robotics, vol. 3, iss. 4, pp. 200-208, 2013.
    [BibTeX] [Abstract] [EPrints]

    The use of concepts is fundamental to human-level cognition, but there remain a number of open questions as to the structures supporting this competence. Specifically, it has been shown that humans use concept prototypes, a flexible means of representing concepts such that it can be used both for categorisation and for similarity judgements. In the context of autonomous robotic agents, the processes by which such concept functionality could be acquired would be particularly useful, enabling flexible knowledge representation and application. This paper seeks to explore this issue of autonomous concept acquisition. By applying a set of structural and operational principles, that support a wide range of cognitive competencies, within a developmental framework, the intention is to explicitly embed the development of concepts into a wider framework of cognitive processing. Comparison with a benchmark concept modelling system shows that the proposed approach can account for a number of features, namely concept-based classification, and its extension to prototype-like functionality.

    @article{lirolem23077,
              volume = {3},
              number = {4},
               month = {April},
              author = {Paul Baxter and Joachim De Greeff and Rachel Wood and Tony Belpaeme},
                note = {Issue cover date: December 2012},
               title = {Modelling concept prototype competencies using a developmental memory model},
           publisher = {De Gruyter/Springer},
                year = {2013},
             journal = {Paladyn, Journal of Behavioral Robotics},
               pages = {200--208},
            keywords = {ARRAY(0x7f785942f200)},
                 url = {http://eprints.lincoln.ac.uk/23077/},
            abstract = {The use of concepts is fundamental to human-level cognition, but there remain a number of open questions as to the structures supporting this competence. Specifically, it has been shown that humans use concept prototypes, a flexible means of representing concepts such that it can be used both for categorisation and for similarity judgements. In the context of autonomous robotic agents, the processes by which such concept functionality could be acquired would be particularly useful, enabling flexible knowledge representation and application. This paper seeks to explore this issue of autonomous concept acquisition. By applying a set of structural and operational principles, that support a wide range of cognitive competencies, within a developmental framework, the intention is to explicitly embed the development of concepts into a wider framework of cognitive processing. Comparison with a benchmark concept modelling system shows that the proposed approach can account for a number of features, namely concept-based classification, and its extension to prototype-like functionality.}
    }
  • N. Bellotto, M. Hanheide, and N. V. de Weghe, “Qualitative design and implementation of human-robot spatial interactions,” in International Conference on Social Robotics (ICSR), 2013.
    [BibTeX] [Abstract] [EPrints]

    Despite the large number of navigation algorithms available for mobile robots, in many social contexts they often exhibit inopportune motion behaviours in proximity of people, often with very "unnatural" movements due to the execution of segmented trajectories or the sudden activation of safety mechanisms (e.g., for obstacle avoidance). We argue that the reason of the problem is not only the difficulty of modelling human behaviours and generating opportune robot control policies, but also the way human-robot spatial interactions are represented and implemented. In this paper we propose a new methodology based on a qualitative representation of spatial interactions, which is both flexible and compact, adopting the well-defined and coherent formalization of Qualitative Trajectory Calculus (QTC). We show the potential of a QTC-based approach to abstract and design complex robot behaviours, where the desired robot’s behaviour is represented together with its actual performance in one coherent approach, focusing on spatial interactions rather than pure navigation problems.

    @inproceedings{lirolem11637,
           booktitle = {International Conference on Social Robotics (ICSR)},
               month = {October},
               title = {Qualitative design and implementation of human-robot spatial interactions},
              author = {Nicola Bellotto and Marc Hanheide and Nico Van de Weghe},
           publisher = {Springer},
                year = {2013},
            keywords = {ARRAY(0x7f7859410e40)},
                 url = {http://eprints.lincoln.ac.uk/11637/},
            abstract = {Despite the large number of navigation algorithms available for mobile robots, in many social contexts they often exhibit inopportune motion behaviours in proximity of people, often with very "unnatural" movements due to the execution of segmented trajectories or the sudden activation of safety mechanisms (e.g., for obstacle avoidance). We argue that the reason of the problem is not only the difficulty of modelling human behaviours and generating opportune robot control policies, but also the way human-robot spatial interactions are represented and implemented.
    In this paper we propose a new methodology based on a qualitative representation of spatial interactions, which is both flexible and compact, adopting the well-defined and coherent formalization of Qualitative Trajectory Calculus (QTC). We show the potential of a QTC-based approach to abstract and design complex robot behaviours, where the desired robot's behaviour is represented together with its actual performance in one coherent approach, focusing on spatial interactions rather than pure navigation problems.}
    }
  • N. Bellotto, “A multimodal smartphone interface for active perception by visually impaired,” in IEEE SMC Int. Workshop on Human-Machine Systems, Cyborgs and Enhancing Devices (HUMASCEND), 2013.
    [BibTeX] [Abstract] [EPrints]

    The diffuse availability of mobile devices, such as smartphones and tablets, has the potential to bring substantial benefits to the people with sensory impairments. The solution proposed in this paper is part of an ongoing effort to create an accurate obstacle and hazard detector for the visually impaired, which is embedded in a hand-held device. In particular, it presents a proof of concept for a multimodal interface to control the orientation of a smartphone’s camera, while being held by a person, using a combination of vocal messages, 3D sounds and vibrations. The solution, which is to be evaluated experimentally by users, will enable further research in the area of active vision with human-in-the-loop, with potential application to mobile assistive devices for indoor navigation of visually impaired people.

    @inproceedings{lirolem11636,
           booktitle = {IEEE SMC Int. Workshop on Human-Machine Systems, Cyborgs and Enhancing Devices (HUMASCEND)},
               month = {October},
               title = {A multimodal smartphone interface for active perception by visually impaired},
              author = {Nicola Bellotto},
           publisher = {IEEE},
                year = {2013},
            keywords = {ARRAY(0x7f785910e820)},
                 url = {http://eprints.lincoln.ac.uk/11636/},
            abstract = {The diffuse availability of mobile devices, such as smartphones and tablets, has the potential to bring substantial benefits to the people with sensory impairments. The solution proposed in this paper is part of an ongoing effort to create an accurate obstacle and hazard detector for the visually impaired, which is embedded in a hand-held device. In particular, it presents a proof of concept for a multimodal interface to control the orientation of a smartphone's camera, while being held by a person, using a combination of vocal messages, 3D sounds and vibrations. The solution, which is to be evaluated experimentally by users, will enable further research in the area of active vision with human-in-the-loop, with potential application to mobile assistive devices for indoor navigation of visually impaired people.}
    }
  • C. Cherino, G. Cielniak, P. Dickinson, and P. Geril, “FUBUTEC-ECEC’2013,” , 2013.
    [BibTeX] [Abstract] [EPrints]

    This edition covers Risk Management, Management Techniques, Production Design Optimization and Video Applications

    @manual{lirolem22903,
               month = {May},
                type = {Documentation},
               title = {FUBUTEC-ECEC'2013},
              author = {Cristina Cherino and Grzegorz Cielniak and Patrick Dickinson and Philippe Geril},
           publisher = {EUROSIS-ETI BVBA},
                year = {2013},
                note = {FUBUTEC'2013, Future Business Technology Conference, June 10-12, 2013, University of Lincoln, Lincoln, UK},
            keywords = {ARRAY(0x7f785944f0c8)},
                 url = {http://eprints.lincoln.ac.uk/22903/},
            abstract = {This edition covers Risk Management, Management Techniques, Production Design Optimization and Video Applications}
    }
  • G. Cielniak, N. Bellotto, and T. Duckett, “Integrating mobile robotics and vision with undergraduate computer science,” IEEE Transactions on Education, vol. 56, iss. 1, pp. 48-53, 2013.
    [BibTeX] [Abstract] [EPrints]

    This paper describes the integration of robotics education into an undergraduate Computer Science curriculum. The proposed approach delivers mobile robotics as well as covering the closely related field of Computer Vision, and is directly linked to the research conducted at the authors? institution. The paper describes the most relevant details of the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile robots. The specific choices are discussed that were made with regard to the mobile platform, software libraries and lab environment. The paper also presents a detailed qualitative and quantitative analysis of student results, including the correlation between student engagement and performance, and discusses the outcomes of this experience.

    @article{lirolem6031,
              volume = {56},
              number = {1},
               month = {February},
              author = {Grzegorz Cielniak and Nicola Bellotto and Tom Duckett},
               title = {Integrating mobile robotics and vision with undergraduate computer science},
           publisher = {The IEEE Education Society},
                year = {2013},
             journal = {IEEE Transactions on Education},
               pages = {48--53},
            keywords = {ARRAY(0x7f78592cce20)},
                 url = {http://eprints.lincoln.ac.uk/6031/},
            abstract = {This paper describes the integration of robotics education into an undergraduate Computer Science curriculum. The proposed approach delivers mobile robotics as well as covering the closely related field of Computer Vision, and is directly linked to the research conducted at the authors? institution. The paper describes the most relevant details of the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile robots. The specific choices are discussed that were made with regard to the mobile platform, software libraries and lab environment. The paper also presents a detailed qualitative and quantitative analysis of student results, including the correlation between student engagement and performance, and discusses the outcomes of this experience.
    }
    }
  • C. Daniel, G. Neumann, O. Kroemer, and J. Peters, “Learning sequential motor tasks,” in IEEE International Conference on Robotics and Automation, 2013, pp. 2626-2632.
    [BibTeX] [Abstract] [EPrints]

    Many real robot applications require the sequential use of multiple distinct motor primitives. This requirement implies the need to learn the individual primitives as well as a strategy to select the primitives sequentially. Such hierarchical learning problems are commonly either treated as one complex monolithic problem which is hard to learn, or as separate tasks learned in isolation. However, there exists a strong link between the robots strategy and its motor primitives. Consequently, a consistent framework is needed that can learn jointly on the level of the individual primitives and the robots strategy. We present a hierarchical learning method which improves individual motor primitives and, simultaneously, learns how to combine these motor primitives sequentially to solve complex motor tasks. We evaluate our method on the game of robot hockey, which is both difficult to learn in terms of the required motor primitives as well as its strategic elements.

    @inproceedings{lirolem25781,
               month = {May},
              author = {C. Daniel and G. Neumann and O. Kroemer and J. Peters},
                note = {cited By 3},
           booktitle = {IEEE International Conference on Robotics and Automation},
               title = {Learning sequential motor tasks},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {2626--2632},
                year = {2013},
            keywords = {ARRAY(0x7f78593a05c0)},
                 url = {http://eprints.lincoln.ac.uk/25781/},
            abstract = {Many real robot applications require the sequential use of multiple distinct motor primitives. This requirement implies the need to learn the individual primitives as well as a strategy to select the primitives sequentially. Such hierarchical learning problems are commonly either treated as one complex monolithic problem which is hard to learn, or as separate tasks learned in isolation. However, there exists a strong link between the robots strategy and its motor primitives. Consequently, a consistent framework is needed that can learn jointly on the level of the individual primitives and the robots strategy. We present a hierarchical learning method which improves individual motor primitives and, simultaneously, learns how to combine these motor primitives sequentially to solve complex motor tasks. We evaluate our method on the game of robot hockey, which is both difficult to learn in terms of the required motor primitives as well as its strategic elements.}
    }
  • M. P. Deisenroth, G. Neumann, and J. Peters, “A survey on policy search for robotics,” Foundations and Trends in Robotics, vol. 2, iss. 1-2, pp. 388-403, 2013.
    [BibTeX] [Abstract] [EPrints]

    Policy search is a subfield in reinforcement learning which focuses on finding good parameters for a given policy parametrization. It is well suited for robotics as it can cope with high-dimensional state and action spaces, one of the main challenges in robot learning. We review recent successes of both model-free and model-based policy search in robot learning. Model-free policy search is a general approach to learn policies based on sampled trajectories. We classify model-free methods based on their policy evaluation strategy, policy update strategy, and exploration strategy and present a unified view on existing algorithms. Learning a policy is often easier than learning an accurate forward model, and, hence, model-free methods are more frequently used in practice. However, for each sampled trajectory, it is necessary to interact with the * Both authors contributed equally. robot, which can be time consuming and challenging in practice. Modelbased policy search addresses this problem by first learning a simulator of the robot?s dynamics from data. Subsequently, the simulator generates trajectories that are used for policy learning. For both modelfree and model-based policy search methods, we review their respective properties and their applicability to robotic systems.

    @article{lirolem28029,
              volume = {2},
              number = {1-2},
               month = {August},
              author = {M. P. Deisenroth and G. Neumann and J. Peters},
               title = {A survey on policy search for robotics},
           publisher = {Now Publishers},
                year = {2013},
             journal = {Foundations and Trends in Robotics},
               pages = {388--403},
            keywords = {ARRAY(0x7f78593ea2a8)},
                 url = {http://eprints.lincoln.ac.uk/28029/},
            abstract = {Policy search is a subfield in reinforcement learning which focuses on
    finding good parameters for a given policy parametrization. It is well
    suited for robotics as it can cope with high-dimensional state and action
    spaces, one of the main challenges in robot learning. We review recent
    successes of both model-free and model-based policy search in robot
    learning.
    Model-free policy search is a general approach to learn policies
    based on sampled trajectories. We classify model-free methods based on
    their policy evaluation strategy, policy update strategy, and exploration
    strategy and present a unified view on existing algorithms. Learning a
    policy is often easier than learning an accurate forward model, and,
    hence, model-free methods are more frequently used in practice. However,
    for each sampled trajectory, it is necessary to interact with the
    * Both authors contributed equally.
    robot, which can be time consuming and challenging in practice. Modelbased
    policy search addresses this problem by first learning a simulator
    of the robot?s dynamics from data. Subsequently, the simulator generates
    trajectories that are used for policy learning. For both modelfree
    and model-based policy search methods, we review their respective
    properties and their applicability to robotic systems.}
    }
  • T. Duckett, M. Hanheide, T. Krajnik, J. P. Fentanes, and C. Dondrup, “Spatio-temporal representation for cognitive control in long-term scenarios,” in International IEEE/EPSRC Workshop on Autonomous Cognitive Robotics, 2013.
    [BibTeX] [Abstract] [EPrints]

    The FP-7 Integrated Project STRANDS [1] is aimed at producing intelligent mobile robots that are able to operate robustly for months in dynamic human environments. To achieve long-term autonomy, the robots would need to understand the environment and how it changes over time. For that, we will have to develop novel approaches to extract 3D shapes, objects, people, and models of activity from sensor data gathered during months of autonomous operation. So far, the environment models used in mobile robotics have been tailored to capture static scenes and environment variations are largely treated as noise. Therefore, utilization of the static models in ever-changing, real world environments is difficult. We propose to represent the environment?s spatio-temporal dynamics by its frequency spectrum.

    @inproceedings{lirolem14893,
           booktitle = {International IEEE/EPSRC Workshop on Autonomous Cognitive Robotics},
               month = {March},
               title = {Spatio-temporal representation for cognitive control in long-term scenarios},
              author = {Tom Duckett and Marc Hanheide and Tomas Krajnik and Jaime Pulido Fentanes and Christian Dondrup},
                year = {2013},
            keywords = {ARRAY(0x7f7859430820)},
                 url = {http://eprints.lincoln.ac.uk/14893/},
            abstract = {The FP-7 Integrated Project STRANDS [1] is aimed at producing intelligent mobile robots that are able to operate robustly for months in dynamic human environments. To achieve long-term autonomy, the robots would need to understand the environment and how it changes over time. For that, we will have to develop novel approaches to extract 3D shapes, objects, people, and models of activity from sensor data gathered during months of autonomous operation.
    So far, the environment models used in mobile robotics have been tailored to capture static scenes and environment variations are largely treated as noise. Therefore, utilization of the static models in ever-changing, real world environments is difficult. We propose to represent the environment?s spatio-temporal dynamics by its frequency spectrum.}
    }
  • T. Duckett and A. Lilienthal, “Editorial,” Robotics and Autonomous Systems, vol. 61, iss. 10, pp. 1049-1050, 2013.
    [BibTeX] [Abstract] [EPrints]

    .

    @article{lirolem12768,
              volume = {61},
              number = {10},
               month = {October},
              author = {Tom Duckett and Achim Lilienthal},
                note = {Selected Papers from the 5th European Conference on Mobile Robots (ECMR 2011)},
               title = {Editorial},
           publisher = {Elsevier for North-Holland / Intelligent Autonomous Systems (IAS) Society},
                year = {2013},
             journal = {Robotics and Autonomous Systems},
               pages = {1049--1050},
            keywords = {ARRAY(0x7f7859109300)},
                 url = {http://eprints.lincoln.ac.uk/12768/},
            abstract = {.}
    }
  • T. Krajnik, M. Nitsche, J. Faigl, M. Mejail, L. Preucil, and T. Duckett, “External localization system for mobile robotics,” in 16th International Conference on Advanced Robotics (ICAR 2013), 2013.
    [BibTeX] [Abstract] [EPrints]

    We present a fast and precise vision-based software intended for multiple robot localization. The core component of the proposed localization system is an efficient method for black and white circular pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision, and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost camera, its core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. We propose a mathematical model of the method that allows to calculate its precision, area of coverage, and processing speed from the camera?s intrinsic parameters and hardware?s processing capacity. The correctness of the presented model and performance of the algorithm in real-world conditions are verified in several experiments. Apart from the method description, we also publish its source code; so, it can be used as an enabling technology for various mobile robotics problems.

    @inproceedings{lirolem12670,
           booktitle = {16th International Conference on Advanced Robotics (ICAR 2013)},
               month = {November},
               title = {External localization system for mobile robotics},
              author = {Tomas Krajnik and Matias Nitsche  and Jan Faigl and Marta Mejail and Libor Preucil and Tom Duckett},
           publisher = {IEEE},
                year = {2013},
             journal = {International Conference on Advanced Robotics, ICAR 2013 (Proceedings)},
            keywords = {ARRAY(0x7f7858f7f5d0)},
                 url = {http://eprints.lincoln.ac.uk/12670/},
            abstract = {We present a fast and precise vision-based software intended for multiple robot localization. The core component of
    the proposed localization system is an efficient method for black and white circular pattern detection. The method is robust to variable lighting conditions, achieves sub-pixel precision, and its computational complexity is independent of the processed image size. With off-the-shelf computational equipment and low-cost camera, its core algorithm is able to process hundreds of images per second while tracking hundreds of objects with millimeter precision. We propose a mathematical model of the method that allows to calculate its precision, area of coverage, and processing speed from the camera?s intrinsic parameters and hardware?s processing capacity. The correctness of the presented model and
    performance of the algorithm in real-world conditions are verified in several experiments. Apart from the method description, we also publish its source code; so, it can be used as an enabling technology for various mobile robotics problems.}
    }
  • A. G. Kupcsik, M. P. Deisenroth, J. Peters, and G. Neumann, “Data-efficient generalization of robot skills with contextual policy search,” Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI 2013, pp. 1401-1407, 2013.
    [BibTeX] [Abstract] [EPrints]

    In robotics, controllers make the robot solve a task within a specific context. The context can describe the objectives of the robot or physical properties of the environment and is always specified before task execution. To generalize the controller to multiple contexts, we follow a hierarchical approach for policy learning: A lower-level policy controls the robot for a given context and an upper-level policy generalizes among contexts. Current approaches for learning such upper-level policies are based on model-free policy search, which require an excessive number of interactions of the robot with its environment. More data-efficient policy search approaches are model based but, thus far, without the capability of learning hierarchical policies. We propose a new model-based policy search approach that can also learn contextual upper-level policies. Our approach is based on learning probabilistic forward models for long-term predictions. Using these redictions, we use information-theoretic insights to improve the upper-level policy. Our method achieves a substantial improvement in learning speed compared to existing methods on simulated and real robotic tasks.

    @article{lirolem25777,
               month = {July},
               title = {Data-efficient generalization of robot skills with contextual policy search},
              author = {A. G. Kupcsik and M. P. Deisenroth and J. Peters and Gerhard Neumann},
                year = {2013},
               pages = {1401--1407},
                note = {27th AAAI Conference on Artificial Intelligence, AAAI 2013; Bellevue, WA; United States; 14 - 18 July 2013},
             journal = {Proceedings of the 27th AAAI Conference on Artificial Intelligence, AAAI 2013},
            keywords = {ARRAY(0x7f7859435980)},
                 url = {http://eprints.lincoln.ac.uk/25777/},
            abstract = {In robotics, controllers make the robot solve a task within a specific context. The context can describe the objectives of
    the robot or physical properties of the environment and is always specified before task execution. To generalize the controller to multiple contexts, we follow a hierarchical approach for policy learning: A lower-level policy controls the robot for a given context and an upper-level policy generalizes among contexts. Current approaches for learning such upper-level policies are based on model-free policy search, which require an excessive number of interactions of the robot with its environment.
    More data-efficient policy search approaches are model based but, thus far, without the capability of learning
    hierarchical policies. We propose a new model-based policy search approach that can also learn contextual upper-level
    policies. Our approach is based on learning probabilistic forward models for long-term predictions. Using these  redictions, we use information-theoretic insights to improve the upper-level policy. Our method achieves a substantial improvement in learning speed compared to existing methods on simulated and real robotic tasks.}
    }
  • C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, “Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection,” in International Conference on Robotics and Automation (ICRA), 2013, pp. 170-177.
    [BibTeX] [Abstract] [EPrints]

    Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.

    @inproceedings{lirolem7880,
               month = {May},
              author = {Christian Lang and Sven Wachsmuth and Marc Hanheide and Heiko Wersing},
                note = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.},
           booktitle = {International Conference on Robotics and Automation (ICRA)},
               title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
           publisher = {IEEE},
               pages = {170--177},
                year = {2013},
            keywords = {ARRAY(0x7f78592ec778)},
                 url = {http://eprints.lincoln.ac.uk/7880/},
            abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in humanrobot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classi?cation procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classi?cation, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classi?ed in this work. The achieved classi?cation accuracies are comparable to the average human recognition performance and outperformed our previous results on this task.}
    }
  • C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, “Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection,” in IEEE International Conference on Robotics and Automation (ICRA) , Karlsruhe, 2013, pp. 170-177.
    [BibTeX] [Abstract] [EPrints]

    Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. Â\copyright 2013 IEEE.

    @inproceedings{lirolem13775,
               month = {May},
              author = {C. Lang and S. Wachsmuth and M. Hanheide and H. Wersing},
                note = { Conference Code:100673},
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA) },
             address = {Karlsruhe},
               title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
           publisher = {IEEE},
                year = {2013},
               pages = {170--177},
            keywords = {ARRAY(0x7f7859415310)},
                 url = {http://eprints.lincoln.ac.uk/13775/},
            abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. {\^A}{\copyright} 2013 IEEE.}
    }
  • C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, “Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection,” in IEEE International Conference on Robotics and Automation, ICRA 2013, Karlsruhe, 2013, pp. 170-177.
    [BibTeX] [Abstract] [EPrints]

    Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. Â\copyright 2013 IEEE.

    @inproceedings{lirolem13462,
              author = {C. Lang and S. Wachsmuth and M. Hanheide and H. Wersing},
                note = {Conference Code:100673},
           booktitle = {IEEE International Conference on Robotics and Automation, ICRA 2013},
             address = {Karlsruhe},
               title = {Facial communicative signal interpretation in human-robot interaction by discriminative video subsequence selection},
           publisher = {IEEE},
               pages = {170--177},
                year = {2013},
            keywords = {ARRAY(0x7f78592c9918)},
                 url = {http://eprints.lincoln.ac.uk/13462/},
            abstract = {Facial communicative signals (FCSs) such as head gestures, eye gaze, and facial expressions can provide useful feedback in conversations between people and also in human-robot interaction. This paper presents a pattern recognition approach for the interpretation of FCSs in terms of valence, based on the selection of discriminative subsequences in video data. These subsequences capture important temporal dynamics and are used as prototypical reference subsequences in a classification procedure based on dynamic time warping and feature extraction with active appearance models. Using this valence classification, the robot can discriminate positive from negative interaction situations and react accordingly. The approach is evaluated on a database containing videos of people interacting with a robot by teaching the names of several objects to it. The verbal answer of the robot is expected to elicit the display of spontaneous FCSs by the human tutor, which were classified in this work. The achieved classification accuracies are comparable to the average human recognition performance and outperformed our previous results on this task. {\^A}{\copyright} 2013 IEEE.}
    }
  • F. Moreno, G. Cielniak, and T. Duckett, “Evaluation of laser range-finder mapping for agricultural spraying vehicles,” in Towards Autonomous Robotic Systems, 2013, pp. 210-221.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop. However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer’s cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.

    @inproceedings{lirolem11330,
           booktitle = {Towards Autonomous Robotic Systems},
               month = {August},
               title = {Evaluation of laser range-finder mapping for agricultural spraying vehicles},
              author = {Francisco-Angel Moreno and Grzegorz Cielniak and Tom Duckett},
                year = {2013},
               pages = {210--221},
            keywords = {ARRAY(0x7f78593f25b8)},
                 url = {http://eprints.lincoln.ac.uk/11330/},
            abstract = {In this paper, we present a new application of laser range-finder sensing to agricultural spraying vehicles. The current generation of spraying vehicles use automatic controllers to maintain the height of the sprayer booms above the crop.
    However, these control systems are typically based on ultrasonic sensors mounted on the booms, which limits the accuracy of the measurements and the response of the controller to changes in the terrain, resulting in a sub-optimal spraying process. To overcome these limitations, we propose to use a laser scanner, attached to the front of the sprayer's cabin, to scan the ground surface in front of the vehicle and to build a scrolling 3d map of the terrain. We evaluate the proposed solution in a series of field tests, demonstrating that the approach provides a more detailed and accurate representation of the environment than the current sonar-based solution, and which can lead to the development of more efficient boom control systems.}
    }
  • A. Paraschos, C. Daniel, J. Peters, and G. Neumann, “Probabilistic movement primitives,” in Advances in Neural Information Processing Systems, (NIPS), 2013.
    [BibTeX] [Abstract] [EPrints]

    Movement Primitives (MP) are a well-established approach for representing modular and re-usable robot movement generators. Many state-of-the-art robot learning successes are based MPs, due to their compact representation of the inherently continuous and high dimensional robot movements. A major goal in robot learning is to combine multiple MPs as building blocks in a modular control architecture to solve complex tasks. To this effect, a MP representation has to allow for blending between motions, adapting to altered task variables, and co-activating multiple MPs in parallel. We present a probabilistic formulation of the MP concept that maintains a distribution over trajectories. Our probabilistic approach allows for the derivation of new operations which are essential for implementing all aforementioned properties in one framework. In order to use such a trajectory distribution for robot movement control, we analytically derive a stochastic feedback controller which reproduces the given trajectory distribution. We evaluate and compare our approach to existing methods on several simulated as well as real robot scenarios.

    @inproceedings{lirolem25785,
           booktitle = {Advances in Neural Information Processing Systems, (NIPS)},
               month = {December},
               title = {Probabilistic movement primitives},
              author = {A. Paraschos and C. Daniel and J. Peters and G. Neumann},
                year = {2013},
             journal = {Advances in Neural Information Processing Systems},
            keywords = {ARRAY(0x7f78592f02d0)},
                 url = {http://eprints.lincoln.ac.uk/25785/},
            abstract = {Movement Primitives (MP) are a well-established approach for representing modular
    and re-usable robot movement generators. Many state-of-the-art robot learning
    successes are based MPs, due to their compact representation of the inherently
    continuous and high dimensional robot movements. A major goal in robot learning
    is to combine multiple MPs as building blocks in a modular control architecture
    to solve complex tasks. To this effect, a MP representation has to allow for
    blending between motions, adapting to altered task variables, and co-activating
    multiple MPs in parallel. We present a probabilistic formulation of the MP concept
    that maintains a distribution over trajectories. Our probabilistic approach
    allows for the derivation of new operations which are essential for implementing
    all aforementioned properties in one framework. In order to use such a trajectory
    distribution for robot movement control, we analytically derive a stochastic feedback
    controller which reproduces the given trajectory distribution. We evaluate
    and compare our approach to existing methods on several simulated as well as
    real robot scenarios.}
    }
  • A. Paraschos, G. Neumann, and J. Peters, “A probabilistic approach to robot trajectory generation,” in 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 2013, pp. 477-483.
    [BibTeX] [Abstract] [EPrints]

    Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations – a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.

    @inproceedings{lirolem25693,
              volume = {2015-F},
              number = {Februa},
               month = {October},
              author = {A. Paraschos and G. Neumann and J. Peters},
           booktitle = {13th IEEE-RAS International Conference on  Humanoid Robots (Humanoids)},
               title = {A probabilistic approach to robot trajectory generation},
           publisher = {IEEE},
                year = {2013},
               pages = {477--483},
            keywords = {ARRAY(0x7f78590b5168)},
                 url = {http://eprints.lincoln.ac.uk/25693/},
            abstract = {Motor Primitives (MPs) are a promising approach for the data-driven acquisition as well as for the modular and re-usable generation of movements. However, a modular control architecture with MPs is only effective if the MPs support co-activation as well as continuously blending the activation from one MP to the next. In addition, we need efficient mechanisms to adapt a MP to the current situation. Common approaches to movement primitives lack such capabilities or their implementation is based on heuristics. We present a probabilistic movement primitive approach that overcomes the limitations of existing approaches. We encode a primitive as a probability distribution over trajectories. The representation as distribution has several beneficial properties. It allows encoding a time-varying variance profile. Most importantly, it allows performing new operations - a product of distributions for the co-activation of MPs conditioning for generalizing the MP to different desired targets. We derive a feedback controller that reproduces a given trajectory distribution in closed form. We compare our approach to the existing state-of-the art and present real robot results for learning from demonstration.}
    }
  • E. A. Rueckert, G. Neumann, M. Toussaint, and W. Maass, “Learned graphical models for probabilistic planning provide a new class of movement primitives,” Frontiers in Computational Neuroscience, vol. 6, 2013.
    [BibTeX] [Abstract] [EPrints]

    Biological movement generation combines three interesting aspects: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.

    @article{lirolem25789,
              volume = {6},
               month = {January},
               title = {Learned graphical models for probabilistic planning provide a new class of movement primitives},
              author = {Elmar A. Rueckert and Gerhard Neumann and Marc Toussaint and Wolfgang Maass},
           publisher = {Frontiers Media},
                year = {2013},
             journal = {Frontiers in Computational Neuroscience},
            keywords = {ARRAY(0x7f78593dec50)},
                 url = {http://eprints.lincoln.ac.uk/25789/},
            abstract = {Biological movement generation combines three interesting aspects: its modular organization in movement primitives (MPs), its characteristics of stochastic optimality under perturbations, and its efficiency in terms of learning. A common approach to motor skill learning is to endow the primitives with dynamical systems. Here, the parameters of the primitive indirectly define the shape of a reference trajectory. We propose an alternative MP representation based on probabilistic inference in learned graphical models with new and interesting properties that complies with salient features of biological movement control. Instead of endowing the primitives with dynamical systems, we propose to endow MPs with an intrinsic probabilistic planning system, integrating the power of stochastic optimal control (SOC) methods within a MP. The parameterization of the primitive is a graphical model that represents the dynamics and intrinsic cost function such that inference in this graphical model yields the control policy. We parameterize the intrinsic cost function using task-relevant features, such as the importance of passing through certain via-points. The system dynamics as well as intrinsic cost function parameters are learned in a reinforcement learning (RL) setting. We evaluate our approach on a complex 4-link balancing task. Our experiments show that our movement representation facilitates learning significantly and leads to better generalization to new task settings without re-learning.}
    }
  • P. S. Teh, A. B. J. Teoh, and S. Yue, “A survey of keystroke dynamics biometrics,” The Scientific World Journal, vol. 2013, p. 408280, 2013.
    [BibTeX] [Abstract] [EPrints]

    Research on keystroke dynamics biometrics has been increasing, especially in the last decade. The main motivation behind this effort is due to the fact that keystroke dynamics biometrics is economical and can be easily integrated into the existing computer security systems with minimal alteration and user intervention. Numerous studies have been conducted in terms of data acquisition devices, feature representations, classification methods, experimental protocols, and evaluations. However, an up-to-date extensive survey and evaluation is not yet available. The objective of this paper is to provide an insightful survey and comparison on keystroke dynamics biometrics research performed throughout the last three decades, as well as offering suggestions and possible future research directions.

    @article{lirolem12817,
              volume = {2013},
               month = {December},
              author = {Pin Shen Teh and Andrew Beng Jin Teoh and Shigang Yue},
               title = {A survey of keystroke dynamics biometrics},
           publisher = {Hindawi Publishing Corporation / Scientific World},
             journal = {The Scientific World Journal},
               pages = {408280},
                year = {2013},
            keywords = {ARRAY(0x7f78593a9908)},
                 url = {http://eprints.lincoln.ac.uk/12817/},
            abstract = {Research on keystroke dynamics biometrics has been increasing, especially in the last decade. The main motivation behind this effort is due to the fact that keystroke dynamics biometrics is economical and can be easily integrated into the existing computer security systems with minimal alteration and user intervention. Numerous studies have been conducted in terms of data acquisition devices, feature representations, classification methods, experimental protocols, and evaluations. However, an up-to-date extensive survey and evaluation is not yet available. The objective of this paper is to provide an insightful survey and comparison on keystroke dynamics biometrics research performed throughout the last three decades, as well as offering suggestions and possible future research directions.}
    }
  • A. Wystrach, M. Mangan, A. Philippides, and P. Graham, “Snapshots in ants? New interpretations of paradigmatic experiments,” Journal of Experimental Biology, vol. 216, iss. 10, pp. 1766-1770, 2013.
    [BibTeX] [Abstract] [EPrints]

    Ants can use visual information to guide long idiosyncratic routes and accurately pinpoint locations in complex natural environments. It has often been assumed that the world knowledge of these foragers consists of multiple discrete views that are retrieved sequentially for breaking routes into sections controlling approaches to a goal. Here we challenge this idea using a model of visual navigation that does not store and use discrete views to replicate the results from paradigmatic experiments that have been taken as evidence that ants navigate using such discrete snapshots. Instead of sequentially retrieving views, the proposed architecture gathers information from all experienced views into a single memory network, and uses this network all along the route to determine the most familiar heading at a given location. This algorithm is consistent with the navigation of ants in both laboratory and natural environments, and provides a parsimonious solution to deal with visual information from multiple locations.

    @article{lirolem23579,
              volume = {216},
              number = {10},
               month = {May},
              author = {Antoine Wystrach and Michael Mangan and Andrew Philippides and Paul Graham},
               title = {Snapshots in ants? New interpretations of paradigmatic experiments},
           publisher = {The Company of Biologists Ltd},
                year = {2013},
             journal = {Journal of Experimental Biology},
               pages = {1766--1770},
            keywords = {ARRAY(0x7f78593a7950)},
                 url = {http://eprints.lincoln.ac.uk/23579/},
            abstract = {Ants can use visual information to guide long idiosyncratic routes and accurately pinpoint locations in complex natural environments. It has often been assumed that the world knowledge of these foragers consists of multiple discrete views that are retrieved sequentially for breaking routes into sections controlling approaches to a goal. Here we challenge this idea using a model of visual navigation that does not store and use discrete views to replicate the results from paradigmatic experiments that have been taken as evidence that ants navigate using such discrete snapshots. Instead of sequentially retrieving views, the proposed architecture gathers information from all experienced views into a single memory network, and uses this network all along the route to determine the most familiar heading at a given location. This algorithm is consistent with the navigation of ants in both laboratory and natural environments, and provides a parsimonious solution to deal with visual information from multiple locations.}
    }
  • J. Xu, S. Yue, and Y. Tang, “A motion attention model based on rarity weighting and motion cues in dynamic scenes,” International Journal of Pattern Recognition and Artificial Intelligence, vol. 27, iss. 06, p. 1355009, 2013.
    [BibTeX] [Abstract] [EPrints]

    Nowadays, motion attention model is a controversial topic in the biological computer vision area. The computational attention model can be decomposed into a set of features via predefined channels. Here we designed a bio-inspired vision attention model, and added the rarity measurement onto it. The priority of rarity is emphasized under the assumption of weighting effect upon the features logic fusion. At this stage, a final saliency map at each frame is adjusted by the spatiotemporal and rarity values. By doing this, the process of mimicking human vision attention becomes more realistic and logical to the real circumstance. The experiments are conducted on the benchmark dataset of static images and video sequences. We simulated the attention shift based on several dataset. Most importantly, our dynamic scenes are mostly selected from the objects moving on the highway and dynamic scenes. The former one can be developed on the detection of car collision and will be a useful tool for further application in robotics. We also conduct experiment on the other video clips to prove the rationality of rarity factor and feature cues fusion methods. Finally, the evaluation results indicate our visual attention model outperforms several state-of-the-art motion attention models. Read More: http://www.worldscientific.com/doi/abs/10.1142/S0218001413550094

    @article{lirolem13793,
              volume = {27},
              number = {06},
               month = {September},
              author = {Jiawei Xu and Shigang Yue and Yuchao Tang},
               title = {A motion attention model based on rarity weighting and motion cues in dynamic scenes},
           publisher = {World Scientific Publishing},
                year = {2013},
             journal = {International Journal of Pattern Recognition and Artificial Intelligence},
               pages = {1355009},
            keywords = {ARRAY(0x7f78592ea3e8)},
                 url = {http://eprints.lincoln.ac.uk/13793/},
            abstract = {Nowadays, motion attention model is a controversial topic in the biological computer vision area. The computational attention model can be decomposed into a set of features via predefined channels. Here we designed a bio-inspired vision attention model, and added the rarity measurement onto it. The priority of rarity is emphasized under the assumption of weighting effect upon the features logic fusion. At this stage, a final saliency map at each frame is adjusted by the spatiotemporal and rarity values. By doing this, the process of mimicking human vision attention becomes more realistic and logical to the real circumstance. The experiments are conducted on the benchmark dataset of static images and video sequences. We simulated the attention shift based on several dataset. Most importantly, our dynamic scenes are mostly selected from the objects moving on the highway and dynamic scenes. The former one can be developed on the detection of car collision and will be a useful tool for further application in robotics. We also conduct experiment on the other video clips to prove the rationality of rarity factor and feature cues fusion methods. Finally, the evaluation results indicate our visual attention model outperforms several state-of-the-art motion attention models.
    
    
    Read More: http://www.worldscientific.com/doi/abs/10.1142/S0218001413550094}
    }
  • S. Yue and C. F. Rind, “Redundant neural vision systems: competing for collision recognition roles,” IEEE Transactions on Autonomous Mental Development, vol. 5, iss. 2, pp. 173-186, 2013.
    [BibTeX] [Abstract] [EPrints]

    Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modelling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems ? the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.

    @article{lirolem9307,
              volume = {5},
              number = {2},
               month = {June},
              author = {Shigang Yue and F. Claire Rind},
               title = {Redundant neural vision systems: competing for collision recognition roles},
           publisher = {IEEE / Institute of Electrical and Electronics Engineers Incorporated},
                year = {2013},
             journal = {IEEE Transactions on Autonomous Mental Development},
               pages = {173--186},
            keywords = {ARRAY(0x7f785941e328)},
                 url = {http://eprints.lincoln.ac.uk/9307/},
            abstract = {Ability to detect collisions is vital for future robots that interact with humans in complex visual environments. Lobula giant movement detectors (LGMD) and directional selective neurons (DSNs) are two types of identified neurons found in the visual pathways of insects such as locusts. Recent modelling studies showed that the LGMD or grouped DSNs could each be tuned for collision recognition. In both biological and artificial vision systems, however, which one should play the collision recognition role and the way the two types of specialized visual neurons could be functioning together are not clear. In this modeling study, we compared the competence of the LGMD and the DSNs, and also investigate the cooperation of the two neural vision systems for collision recognition via artificial evolution. We implemented three types of collision recognition neural subsystems ? the LGMD, the DSNs and a hybrid system which combines the LGMD and the DSNs subsystems together, in each individual agent. A switch gene determines which of the three redundant neural subsystems plays the collision recognition role. We found that, in both robotics and driving environments, the LGMD was able to build up its ability for collision recognition quickly and robustly therefore reducing the chance of other types of neural networks to play the same role. The results suggest that the LGMD neural network could be the ideal model to be realized in hardware for collision recognition.}
    }
  • S. Yue and C. F. Rind, “Postsynaptic organizations of directional selective visual neural networks for collision detection,” Neurocomputing, vol. 103, pp. 50-62, 2013.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we studied the postsynaptic organizations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organizations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organizations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94\% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organizations can play.

    @article{lirolem9308,
              volume = {103},
               month = {March},
              author = {Shigang Yue and F. Claire Rind},
               title = {Postsynaptic organizations of directional selective visual neural networks for collision detection},
           publisher = {Elsevier Science Limited},
             journal = {Neurocomputing},
               pages = {50--62},
                year = {2013},
            keywords = {ARRAY(0x7f78593a7da0)},
                 url = {http://eprints.lincoln.ac.uk/9308/},
            abstract = {In this paper, we studied the postsynaptic organizations of directional selective visual neurons for collision detection. Directional selective neurons can extract different directional visual motion cues fast and reliably by allowing inhibition spreads to further layers in specific directions with one or several time steps delay. Whether these directional selective neurons can be easily organised for other specific visual tasks is not known. Taking collision detection as the primary visual task, we investigated the postsynaptic organizations of these directional selective neurons through evolutionary processes. The evolved postsynaptic organizations demonstrated robust properties in detecting imminent collisions in complex visual environments with many of which achieved 94\% success rate after evolution suggesting active roles in collision detection directional selective neurons and its postsynaptic organizations can play. }
    }
  • G. Zahi and S. Yue, “Automatic detection of low light images in a video sequence Shot under different light conditions,” in Modelling Symposium (EMS), 2013 European, 2013, pp. 271-276.
    [BibTeX] [Abstract] [EPrints]

    Nocturnal insects have the ability to neurally sum visual signals in space and time to be able to see under very low light conditions. This ability shown by nocturnal insects has inspired many researchers to develop a night vision algorithm, that is capable of significantly improving the quality and reliability of digital images captured under very low light conditions. This algorithm however when applied to day time images rather degrades their quality. It is therefore not suitable to apply the night vision algorithms equally to an image stream with different light conditions. This paper introduces a quick method of automatically determining when to apply the nocturnal vision algorithm by analysing the cumulative intensity histogram of each image in the stream. The effectiveness of this method is demonstrated with relevant experiments in a good and acceptable way.

    @inproceedings{lirolem13757,
           booktitle = {Modelling Symposium (EMS), 2013 European},
               month = {November},
               title = {Automatic detection of low light images in a video sequence Shot under different light conditions},
              author = {Gabriel Zahi and Shigang Yue},
           publisher = {IEEE},
                year = {2013},
               pages = {271--276},
            keywords = {ARRAY(0x7f78592d6b08)},
                 url = {http://eprints.lincoln.ac.uk/13757/},
            abstract = {Nocturnal insects have the ability to neurally sum visual signals in space and time to be able to see under very low light conditions. This ability shown by nocturnal insects has inspired many researchers to develop a night vision algorithm, that is capable of significantly improving the quality and reliability of digital images captured under very low light conditions. This algorithm however when applied to day time images rather degrades their quality. It is therefore not suitable to apply the night vision algorithms equally to an image stream with different light conditions. This paper introduces a quick method of automatically determining when to apply the nocturnal vision algorithm by analysing the cumulative intensity histogram of each image in the stream. The effectiveness of this method is demonstrated with relevant experiments in a good and acceptable way.}
    }
  • M. Zillich, K. Zhou, D. Skocaj, M. Kristan, A. Vrecko, M. Mahnic, M. Janicek, G. M. Kruijff, T. Keller, M. Hanheide, and N. Hawes, “Robot George: interactive continuous learning of visual concepts,” in Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction, 2013, p. 425.
    [BibTeX] [Abstract] [EPrints]

    The video presents the robot George learning visual concepts in dialogue with a tutor

    @inproceedings{lirolem8365,
           booktitle = {Proceedings of the 8th ACM/IEEE international conference on Human-robot interaction},
               month = {March},
               title = {Robot George: interactive continuous learning of visual concepts},
              author = {Michael Zillich and Kai Zhou and Danijel Skocaj and Matej Kristan and Alen Vrecko and Marko Mahnic and Miroslav Janicek and Geert-Jan M. Kruijff and Thomas Keller and Marc Hanheide and Nick Hawes},
           publisher = {IEEE Press},
                year = {2013},
               pages = {425},
            keywords = {ARRAY(0x7f78592dd038)},
                 url = {http://eprints.lincoln.ac.uk/8365/},
            abstract = {The video presents the robot George learning visual concepts in dialogue with a tutor}
    }

2012

  • H. B. Amor, O. Kroemer, U. Hillenbrand, G. Neumann, and J. Peters, “Generalization of human grasping for multi-fingered robot hands,” in International Conference on Robot Systems (IROS), 2012, pp. 2043-2050.
    [BibTeX] [Abstract] [EPrints]

    Multi-fingered robot grasping is a challenging problem that is difficult to tackle using hand-coded programs. In this paper we present an imitation learning approach for learning and generalizing grasping skills based on human demonstrations. To this end, we split the task of synthesizing a grasping motion into three parts: (1) learning efficient grasp representations from human demonstrations, (2) warping contact points onto new objects, and (3) optimizing and executing the reach-and-grasp movements. We learn low-dimensional latent grasp spaces for different grasp types, which form the basis for a novel extension to dynamic motor primitives. These latent-space dynamic motor primitives are used to synthesize entire reach-and-grasp movements. We evaluated our method on a real humanoid robot. The results of the experiment demonstrate the robustness and versatility of our approach.

    @inproceedings{lirolem25788,
           booktitle = {International Conference on Robot Systems (IROS)},
               month = {December},
               title = {Generalization of human grasping for multi-fingered robot hands},
              author = {Heni Ben Amor and Oliver Kroemer and Ulrich Hillenbrand and Gerhard Neumann and Jan Peters},
                year = {2012},
               pages = {2043--2050},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
            keywords = {ARRAY(0x7f785940bad0)},
                 url = {http://eprints.lincoln.ac.uk/25788/},
            abstract = {Multi-fingered robot grasping is a challenging
    problem that is difficult to tackle using hand-coded programs.
    In this paper we present an imitation learning approach for
    learning and generalizing grasping skills based on human
    demonstrations. To this end, we split the task of synthesizing
    a grasping motion into three parts: (1) learning efficient grasp
    representations from human demonstrations, (2) warping contact
    points onto new objects, and (3) optimizing and executing
    the reach-and-grasp movements. We learn low-dimensional
    latent grasp spaces for different grasp types, which form the
    basis for a novel extension to dynamic motor primitives. These
    latent-space dynamic motor primitives are used to synthesize
    entire reach-and-grasp movements. We evaluated our method
    on a real humanoid robot. The results of the experiment
    demonstrate the robustness and versatility of our approach.}
    }
  • F. Arvin, A. E. Turgut, and S. Yue, “Fuzzy-based aggregation with a mobile robot swarm,” Lecture Notes in Computer Science, vol. 7461, pp. 346-347, 2012.
    [BibTeX] [Abstract] [EPrints]

    Aggregation is a widely observed phenomenon in social insects and animals such as cockroaches, honeybees and birds. From swarm robotics perspective [3], aggregation can be defined as gathering randomly distributed robots to form an aggregate. Honeybee aggregation is an example of cue-based aggregation method that was studied in [4]. In that study, micro robots were deployed in a gradually lighted environment to mimic the behavior of honeybees which aggregate around a zone that has the optimal temperature (BEECLUST). In our previous study [2], two modifications on BEECLUST ? dynamic velocity and comparative waiting time ? were applied to increase the performance of aggregation.

    @article{lirolem7328,
              volume = {7461},
               month = {September},
              author = {Farshad Arvin and Ali Emre Turgut and Shigang Yue},
               title = {Fuzzy-based aggregation with a mobile robot swarm},
           publisher = {Springer},
             journal = {Lecture Notes in Computer Science},
               pages = {346--347},
                year = {2012},
            keywords = {ARRAY(0x7f78590bf898)},
                 url = {http://eprints.lincoln.ac.uk/7328/},
            abstract = {Aggregation is a widely observed phenomenon in social insects and animals such as cockroaches, honeybees and birds. From swarm robotics perspective [3], aggregation can be defined as gathering randomly distributed robots to form an aggregate. Honeybee aggregation is an example of cue-based aggregation method that was studied in [4]. In that study, micro robots were deployed in a gradually lighted environment to mimic the behavior of honeybees which aggregate around a zone that has the optimal temperature (BEECLUST). In our previous study [2], two modifications on BEECLUST ? dynamic velocity and comparative waiting time ? were applied to increase the performance of aggregation.}
    }
  • M. Barnes, M. Dudbridge, and T. Duckett, “Polarised light stress analysis and laser scatter imaging for non-contact inspection of heat seals in food trays,” Journal of Food Engineering, vol. 112, iss. 3, pp. 183-190, 2012.
    [BibTeX] [Abstract] [EPrints]

    This paper introduces novel non-contact methods for detecting faults in heat seals of food packages. Two alternative imaging technologies are investigated; laser scatter imaging and polarised light stress images. After segmenting the seal area from the rest of the respective image, a classifier is trained to detect faults in different regions of the seal area using features extracted from the pixels in the respective region. A very large set of candidate features, based on statistical information relating to the colour and texture of each region, is first extracted. Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating faults from non-faults. With this approach, different features can be selected and optimised for the different imaging methods. In experiments we compare the performance of classifiers trained using features extracted from laser scatter images only, polarised light stress images only, and a combination of both image types. The results show that the polarised light and laser scatter classifiers achieved accuracies of 96$\backslash$\% and 90$\backslash$\%, respectively, while the combination of both sensors achieved an accuracy of 95$\backslash$\%. These figures suggest that both systems have potential for commercial development.

    @article{lirolem5513,
              volume = {112},
              number = {3},
               month = {October},
              author = {Michael Barnes and Michael Dudbridge and Tom Duckett},
               title = {Polarised light stress analysis and laser scatter imaging for non-contact inspection of heat seals in food trays},
           publisher = {Elsevier},
                year = {2012},
             journal = {Journal of Food Engineering},
               pages = {183--190},
            keywords = {ARRAY(0x7f7859458d38)},
                 url = {http://eprints.lincoln.ac.uk/5513/},
            abstract = {This paper introduces novel non-contact methods for detecting faults in heat seals of food packages. Two alternative imaging technologies are investigated; laser scatter imaging and polarised light stress images. After segmenting the seal area from the rest of the respective image, a classifier is trained to detect faults in different regions of the seal area using features extracted from the pixels in the respective region. A very large set of candidate features, based on statistical information relating to the colour and texture of each region, is first extracted. Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating faults from non-faults. With this approach, different features can be selected and optimised for the different imaging methods. In experiments we compare the performance of classifiers trained using features extracted from laser scatter images only, polarised light stress images only, and a combination of both image types. The results show that the polarised light and laser scatter classifiers achieved accuracies of 96{$\backslash$}\% and 90{$\backslash$}\%, respectively, while the combination of both sensors achieved an accuracy of 95{$\backslash$}\%. These figures suggest that both systems have potential for commercial development.}
    }
  • N. Bellotto, B. Benfold, H. Harland, H. Nagel, N. Pirlo, I. Reid, E. Sommerlade, and C. Zhao, “Cognitive visual tracking and camera control,” Computer Vision and Image Understanding, vol. 116, iss. 3, pp. 457-471, 2012.
    [BibTeX] [Abstract] [EPrints]

    Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.

    @article{lirolem4823,
              volume = {116},
              number = {3},
               month = {March},
              author = {Nicola Bellotto and Ben Benfold and Hanno Harland and Hans-Hellmut Nagel and Nicola Pirlo and Ian Reid and Eric Sommerlade and Chuan Zhao},
               title = {Cognitive visual tracking and camera control},
           publisher = {Elsevier},
                year = {2012},
             journal = {Computer Vision and Image Understanding},
               pages = {457--471},
            keywords = {ARRAY(0x7f78594359c8)},
                 url = {http://eprints.lincoln.ac.uk/4823/},
            abstract = {Cognitive visual tracking is the process of observing and understanding the behaviour of a moving person. This paper presents an efficient solution to extract, in real-time, high-level information from an observed scene, and generate the most appropriate commands for a set of pan-tilt-zoom (PTZ) cameras in a surveillance scenario. Such a high-level feedback control loop, which is the main novelty of our work, will serve to reduce uncertainties in the observed scene and to maximize the amount of information extracted from it. It is implemented with a distributed camera system using SQL tables as virtual communication channels, and Situation Graph Trees for knowledge representation, inference and high-level camera control. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.}
    }
  • N. Bellotto, “Robot control based on qualitative representation of human trajectories,” in AAAI Spring Symposium, "Designing Intelligent Robots: Reintegrating AI", 2012.
    [BibTeX] [Abstract] [EPrints]

    A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.

    @inproceedings{lirolem4780,
           booktitle = { AAAI Spring Symposium, "Designing Intelligent Robots: Reintegrating AI"},
               month = {March},
               title = {Robot control based on qualitative representation of human trajectories},
              author = {Nicola Bellotto},
           publisher = {AAAI - Association for the Advancement of Artificial Intelligence},
                year = {2012},
                note = {A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.},
            keywords = {ARRAY(0x7f7859124138)},
                 url = {http://eprints.lincoln.ac.uk/4780/},
            abstract = {A major challenge for future social robots is the high-level interpretation of human motion, and the consequent generation of appropriate robot actions. This paper describes some fundamental steps towards the real-time implementation of a system that allows a mobile robot to transform quantitative information about human trajectories (i.e. coordinates and speed) into qualitative concepts, and from these to generate appropriate control commands. The problem is formulated using a simple version of qualitative trajectory calculus, then solved using an inference engine based on fuzzy temporal logic and situation graph trees. Preliminary results are discussed and future directions of the current research are drawn.}
    }
  • T. Belpaeme, P. Baxter, R. Read, R. Wood, H. Cuayáhuitl, B. Kiefer, S. Racioppa, I. Kruijff-Korbayová, G. Athanasopoulos, V. Enescu, R. Looije, M. Neerincx, Y. Demiris, R. Ros-Espinoza, A. Beck, L. Cañamero, A. Hiolle, M. Lewis, I. Baroni, M. Nalin, P. Cosi, G. Paci, F. Tesser, G. Sommavilla, and R. Humbert, “Multimodal child-robot interaction: building social bonds,” Journal of Human-Robot Interaction, vol. 1, iss. 2, 2012.
    [BibTeX] [Abstract] [EPrints]

    For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competencies and integrating them to form an autonomous robotic system for evaluation ?in the wild.? The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.

    @article{lirolem22210,
              volume = {1},
              number = {2},
               month = {December},
              author = {Tony Belpaeme and Paul Baxter and Robin Read and Rachel Wood and Heriberto Cuay{\'a}huitl and Bernd Kiefer and Stefania Racioppa and Ivana Kruijff-Korbayov{\'a} and Georgios Athanasopoulos and Valentin Enescu and Rosemarijn Looije and Mark Neerincx and Yiannis Demiris and Raquel Ros-Espinoza and Aryel Beck and Lola Ca{\~n}amero and Antione Hiolle and Matthew Lewis and Ilaria Baroni and Marco Nalin and Piero Cosi and Giulio Paci and Fabio Tesser and Giacomo Sommavilla and Remi Humbert},
               title = {Multimodal child-robot interaction: building social bonds},
           publisher = {Clear Facts Research},
             journal = {Journal of Human-Robot Interaction},
                year = {2012},
            keywords = {ARRAY(0x7f785945ed80)},
                 url = {http://eprints.lincoln.ac.uk/22210/},
            abstract = {For robots to interact effectively with human users they must be capable of coordinated, timely behavior in response to social context. The Adaptive Strategies for Sustainable Long-Term Social Interaction (ALIZ-E) project focuses on the design of long-term, adaptive social interaction between robots and child users in real-world settings. In this paper, we report on the iterative approach taken to scientific and technical developments toward this goal: advancing individual technical competencies and integrating them to form an autonomous robotic system for evaluation ?in the wild.? The first evaluation iterations have shown the potential of this methodology in terms of adaptation of the robot to the interactant and the resulting influences on engagement. This sets the foundation for an ongoing research program that seeks to develop technologies for social robot companions.}
    }
  • J. Bird, T. Feltwell, and G. Cielniak, “Real-time adaptive track generation in racing games,” in GAMEON ‘2012, 2012, pp. 17-24.
    [BibTeX] [Abstract] [EPrints]

    Real-time Adaptive Track Generation in Racing Games

    @inproceedings{lirolem6900,
               month = {November},
              author = {Jake Bird and Tom Feltwell and Grzegorz Cielniak},
                note = {Real-time Adaptive Track Generation in Racing Games},
           booktitle = {GAMEON '2012},
               title = {Real-time adaptive track generation in racing games},
           publisher = {Eurosis},
               pages = {17--24},
                year = {2012},
            keywords = {ARRAY(0x7f7858ed03c0)},
                 url = {http://eprints.lincoln.ac.uk/6900/},
            abstract = {Real-time Adaptive Track Generation in Racing Games}
    }
  • G. Cielniak, N. Bellotto, and T. Duckett, “Integrating vision and robotics into the computer science curriculum,” in 3rd International Workshop Teaching Robotics Teaching with Robotics: Integrating Robotics in School Curriculum, 2012.
    [BibTeX] [Abstract] [EPrints]

    This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.

    @inproceedings{lirolem5516,
           booktitle = {3rd International Workshop Teaching Robotics Teaching with Robotics: Integrating Robotics in School Curriculum},
               month = {April},
               title = {Integrating vision and robotics into the computer science curriculum},
              author = {Grzegorz Cielniak and Nicola Bellotto and Tom Duckett},
                year = {2012},
                note = {This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.},
            keywords = {ARRAY(0x7f78592dd8c0)},
                 url = {http://eprints.lincoln.ac.uk/5516/},
            abstract = {This paper describes our efforts in integrating Robotics education into the undergraduate Computer Science curriculum. Our approach delivers Mobile Robotics together with the closely related field of Computer Vision and is directly linked to the research conducted at our institution. The paper describes the most relevant details related to the module content and assessment strategy, paying particular attention to the practical sessions using Rovio mobile webcams. We discuss the specific choices made with regard to the mobile platform, software libraries and lab environment. We also present a detailed qualitative and quantitative analysis, including the correlation between student engagement and performance, and discuss the outcomes of this experience.}
    }
  • C. Daniel, G. Neumann, and J. Peters, “Learning concurrent motor skills in versatile solution spaces,” in International Conference on Intelligent Robot Systems (IROS), 2012, pp. 3591-3597.
    [BibTeX] [Abstract] [EPrints]

    Future robots need to autonomously acquire motor skills in order to reduce their reliance on human programming. Many motor skill learning methods concentrate on learning a single solution for a given task. However, discarding information about additional solutions during learning unnecessarily limits autonomy. Such favoring of single solutions often requires re-learning of motor skills when the task, the environment or the robot?s body changes in a way that renders the learned solution infeasible. Future robots need to be able to adapt to such changes and, ideally, have a large repertoire of movements to cope with such problems. In contrast to current methods, our approach simultaneously learns multiple distinct solutions for the same task, such that a partial degeneration of this solution space does not prevent the successful completion of the task. In this paper, we present a complete framework that is capable of learning different solution strategies for a real robot Tetherball task.

    @inproceedings{lirolem25787,
           booktitle = {International Conference on Intelligent Robot Systems (IROS)},
               month = {October},
               title = {Learning concurrent motor skills in versatile solution spaces},
              author = {C. Daniel and G. Neumann and J. Peters},
                year = {2012},
               pages = {3591--3597},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
            keywords = {ARRAY(0x7f785942c128)},
                 url = {http://eprints.lincoln.ac.uk/25787/},
            abstract = {Future robots need to autonomously acquire motor
    skills in order to reduce their reliance on human programming.
    Many motor skill learning methods concentrate
    on learning a single solution for a given task. However, discarding
    information about additional solutions during learning
    unnecessarily limits autonomy. Such favoring of single solutions
    often requires re-learning of motor skills when the task, the
    environment or the robot?s body changes in a way that renders
    the learned solution infeasible. Future robots need to be able to
    adapt to such changes and, ideally, have a large repertoire of
    movements to cope with such problems. In contrast to current
    methods, our approach simultaneously learns multiple distinct
    solutions for the same task, such that a partial degeneration of
    this solution space does not prevent the successful completion
    of the task. In this paper, we present a complete framework
    that is capable of learning different solution strategies for a
    real robot Tetherball task.}
    }
  • C. Daniel, G. Neumann, and J. Peters, “Hierarchical relative entropy policy search,” in Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS) 2012, 2012, pp. 273-281.
    [BibTeX] [Abstract] [EPrints]

    Many real-world problems are inherently hierarchically structured. The use of this structure in an agent?s policy may well be the key to improved scalability and higher performance. However, such hierarchical structures cannot be exploited by current policy search algorithms. We will concentrate on a basic, but highly relevant hierarchy — the ?mixed option? policy. Here, a gating network first decides which of the options to execute and, subsequently, the option-policy determines the action. In this paper, we reformulate learning a hierarchical policy as a latent variable estimation problem and subsequently extend the Relative Entropy Policy Search (REPS) to the latent variable case. We show that our Hierarchical REPS can learn versatile solutions while also showing an increased performance in terms of learning speed and quality of the found policy in comparison to the nonhierarchical approach.

    @inproceedings{lirolem25791,
              volume = {22},
               month = {April},
              author = {Christian Daniel and Gerhard Neumann and Jan Peters},
           booktitle = {Proceedings of the 15th International Conference on Artificial Intelligence and Statistics (AISTATS) 2012},
               title = {Hierarchical relative entropy policy search},
           publisher = {MIT Press},
               pages = {273--281},
                year = {2012},
            keywords = {ARRAY(0x7f7859462140)},
                 url = {http://eprints.lincoln.ac.uk/25791/},
            abstract = {Many real-world problems are inherently hierarchically
    structured. The use of this structure
    in an agent?s policy may well be the
    key to improved scalability and higher performance.
    However, such hierarchical structures
    cannot be exploited by current policy
    search algorithms. We will concentrate on
    a basic, but highly relevant hierarchy {--} the
    ?mixed option? policy. Here, a gating network
    first decides which of the options to execute
    and, subsequently, the option-policy determines
    the action.
    In this paper, we reformulate learning a hierarchical
    policy as a latent variable estimation
    problem and subsequently extend the
    Relative Entropy Policy Search (REPS) to
    the latent variable case. We show that our
    Hierarchical REPS can learn versatile solutions
    while also showing an increased performance
    in terms of learning speed and quality
    of the found policy in comparison to the nonhierarchical
    approach.}
    }
  • T. Feltwell, P. Dickinson, and G. Cielniak, “A framework for quantitative analysis of user-generated spatial data,” in GAMEON ‘2012, 2012, pp. 17-24.
    [BibTeX] [Abstract] [EPrints]

    This paper proposes a new framework for automated analysis of game-play metrics for aiding game designers in finding out the critical aspects of the game caused by factors like design modications, change in playing style, etc. The core of the algorithm measures similarity between spatial distribution of user generated in-game events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on a data set collected from a modern, multiplayer First Person Shooter, together with application examples of its use. The proposed framework can be used to accompany traditional testing tools and make the game design process more efficient.

    @inproceedings{lirolem6889,
               month = {November},
              author = {Tom Feltwell and Patrick Dickinson and Grzegorz Cielniak},
                note = {This paper proposes a new framework for automated
    analysis of game-play metrics for aiding game designers
    in finding out the critical aspects of the game caused
    by factors like design modications, change in playing
    style, etc. The core of the algorithm measures similarity
    between spatial distribution of user generated in-game
    events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on
    a data set collected from a modern, multiplayer First
    Person Shooter, together with application examples of
    its use. The proposed framework can be used to accompany traditional testing tools and make the game design
    process more efficient.},
           booktitle = {GAMEON '2012},
               title = {A framework for quantitative analysis of user-generated spatial data},
           publisher = {Eurosis},
               pages = {17--24},
                year = {2012},
            keywords = {ARRAY(0x7f78593a55f0)},
                 url = {http://eprints.lincoln.ac.uk/6889/},
            abstract = {This paper proposes a new framework for automated
    analysis of game-play metrics for aiding game designers
    in finding out the critical aspects of the game caused
    by factors like design modications, change in playing
    style, etc. The core of the algorithm measures similarity
    between spatial distribution of user generated in-game
    events and automatically ranks them in order of importance. The feasibility of the method is demonstrated on
    a data set collected from a modern, multiplayer First
    Person Shooter, together with application examples of
    its use. The proposed framework can be used to accompany traditional testing tools and make the game design
    process more efficient.}
    }
  • S. Ghidoni, G. Cielniak, and E. Menegatti, “Texture-based crowd detection and localisation,” in The 12th International Conference on Intelligent Autonomous Systems, 2012, pp. 725-736.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a crowd detection system based on texture analysis. The state-of-the-art techniques based on co-occurrence matrix have been revisited and a novel set of features proposed. These features provide a richer description of the co-occurrence matrix, and can be exploited to obtain stronger classification results, especially when smaller portions of the image are considered. This is extremely useful for crowd localisation: acquired images are divided into smaller regions in order to perform a classification on each one. A thorough evaluation of the proposed system on a real world data set is also presented: this validates the improvements in reliability of the crowd detection and localisation.

    @inproceedings{lirolem5935,
           booktitle = {The 12th International Conference on Intelligent Autonomous Systems},
               month = {June},
               title = {Texture-based crowd detection and localisation},
              author = {Stefano Ghidoni and Grzegorz Cielniak and Emanuele Menegatti},
           publisher = {IEEE / Robotics and Automation Society},
                year = {2012},
               pages = {725--736},
            keywords = {ARRAY(0x7f7859464308)},
                 url = {http://eprints.lincoln.ac.uk/5935/},
            abstract = {This paper presents a crowd detection system based on texture analysis. The state-of-the-art techniques based on co-occurrence matrix have been revisited and a novel set of features proposed. These features provide a richer description of the co-occurrence matrix, and can be exploited to obtain stronger classification results, especially when smaller portions of the image are considered. This is extremely useful for crowd localisation: acquired images are divided into smaller regions in order to perform a classification on each one. A thorough evaluation of the proposed system on a real world data set is also presented: this validates the improvements in reliability of the crowd detection and localisation.}
    }
  • M. Hanheide, A. Peters, and N. Bellotto, “Analysis of human-robot spatial behaviour applying a qualitative trajectory calculus,” in 21st IEEE International Symposium on Robot and Human Interactive Communication, 2012, pp. 689-694.
    [BibTeX] [Abstract] [EPrints]

    The analysis and understanding of human-robot joint spatial behaviour (JSB) such as guiding, approaching, departing, or coordinating movements in narrow spaces and its communicative and dynamic aspects are key requirements on the road towards more intuitive interaction, safe encounter, and appealing living with mobile robots. This endeavours demand for appropriate models and methodologies to represent JSB and facilitate its analysis. In this paper, we adopt a qualitative trajectory calculus (QTC) as a formal foundation for the analysis and representation of such spatial behaviour of a human and a robot based on a compact encoding of the relative trajectories of two interacting agents in a sequential model. We present this QTC together with a distance measure and a probabilistic behaviour model and outline its usage in an actual JSB study.We argue that the proposed QTC coding scheme and derived methodologies for analysis and modelling are flexible and extensible to be adapted for a variety of other scenarios and studies. I.

    @inproceedings{lirolem6750,
               month = {September},
              author = {Marc Hanheide and Annika Peters and Nicola Bellotto},
           booktitle = {21st IEEE International Symposium on Robot and Human Interactive Communication},
              editor = {B. Gottfried and H. Aghajan},
               title = {Analysis of human-robot spatial behaviour applying a qualitative trajectory calculus},
           publisher = {IEEE},
               pages = {689--694},
                year = {2012},
            keywords = {ARRAY(0x7f785944eb40)},
                 url = {http://eprints.lincoln.ac.uk/6750/},
            abstract = {The analysis and understanding of human-robot joint spatial behaviour (JSB) such as guiding, approaching, departing, or coordinating movements in narrow spaces and its communicative and dynamic aspects are key requirements on the road towards more intuitive interaction, safe encounter, and appealing living with mobile robots. This endeavours demand for appropriate models and methodologies to represent JSB and facilitate its analysis. In this paper, we adopt a qualitative trajectory calculus (QTC) as a formal foundation for the analysis and representation of such spatial behaviour of a human and a robot based on a compact encoding of the relative trajectories of two interacting agents in a sequential model. We present this QTC together with a distance measure and a probabilistic behaviour model and outline its usage in an actual JSB study.We argue that the proposed QTC coding scheme and derived methodologies for analysis and modelling are flexible and extensible to be adapted for a variety of other scenarios and studies. I.}
    }
  • M. Hanheide, M. Lohse, and H. Zender, “Expectations, intentions, and actions in human-robot interaction,” Internation Journal of Social Robotics, vol. 4, iss. 2, pp. 107-108, 2012.
    [BibTeX] [Abstract] [EPrints]

    From the issue entitled "Expectations, Intentions & Actions" Human-robot interaction is becoming increasingly complex through the growing number of abilities, both cognitive and physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and actions. Vice versa, robots lack understanding of the users? expectations, intentions, actions, and social signals.

    @article{lirolem6562,
              volume = {4},
              number = {2},
               month = {April},
              author = {M. Hanheide and M. Lohse and H. Zender},
                note = {From the issue entitled "Expectations, Intentions \& Actions"
    Human-robot interaction is becoming increasingly complex
    through the growing number of abilities, both cognitive and
    physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and
    actions. Vice versa, robots lack understanding of the users?
    expectations, intentions, actions, and social signals.},
               title = {Expectations, intentions, and actions in human-robot interaction},
           publisher = {Springer},
                year = {2012},
             journal = {Internation Journal of Social Robotics},
               pages = {107--108},
            keywords = {ARRAY(0x7f78594359e0)},
                 url = {http://eprints.lincoln.ac.uk/6562/},
            abstract = {From the issue entitled "Expectations, Intentions \& Actions"
    Human-robot interaction is becoming increasingly complex
    through the growing number of abilities, both cognitive and
    physical, available to today?s robots. At the same time, interaction is still often dif?cult because the users do not understand the robots? internal states, expectations, intentions, and
    actions. Vice versa, robots lack understanding of the users?
    expectations, intentions, actions, and social signals.}
    }
  • J. Hutton, G. Harper, and T. Duckett, “A prototype low-cost machine vision system for automatic identification and quantification of potato defects,” in The Dundee Conference – Crop Protection in Northern Britain 2012, 2012, pp. 273-278.
    [BibTeX] [Abstract] [EPrints]

    This paper reports on a current project to develop a prototype system for the automatic identification and quantification of potato defects based on machine vision. The system developed uses off-the-shelf hardware, including a low-cost vision sensor and a standard desktop computer with a graphics processing unit (GPU), together with software algorithms to enable detection, identification and quantification of common defects affecting potatoes at near-real-time frame rates. The system uses state-of-the-art image processing and machine learning techniques to automatically learn the appearance of different defect types. It also incorporates an intuitive graphical user interface (GUI) to enable easy set-up of the system by quality control (QC) staff working in the industry.

    @inproceedings{lirolem14511,
           booktitle = {The  Dundee Conference  - Crop  Protection  in  Northern  Britain  2012},
               month = {February},
               title = {A prototype low-cost machine vision system for automatic identification and quantification of potato defects},
              author = {Jamie Hutton and Glyn Harper and Tom Duckett},
           publisher = {Proceedings Crop Protection in Northern Britain 2012},
                year = {2012},
               pages = {273--278},
            keywords = {ARRAY(0x7f78592eea18)},
                 url = {http://eprints.lincoln.ac.uk/14511/},
            abstract = {This paper reports on a current project to develop a prototype system
    for the automatic identification and quantification of potato defects based on
    machine vision. The system developed uses off-the-shelf hardware, including a
    low-cost vision sensor and a standard desktop computer with a graphics processing
    unit (GPU), together with software algorithms to enable detection, identification
    and quantification of common defects affecting potatoes at near-real-time frame
    rates. The system uses state-of-the-art image processing and machine learning
    techniques to automatically learn the appearance of different defect types. It also
    incorporates an intuitive graphical user interface (GUI) to enable easy set-up of the
    system by quality control (QC) staff working in the industry.}
    }
  • C. Jayne, S. Yue, and L. Iliadis, Engineering applications of neural networks, Heidelberg: Springer, 2012, vol. 311.
    [BibTeX] [Abstract] [EPrints]

    Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012

    @book{lirolem7434,
              volume = {311},
              author = {Crisina Jayne and Shigang Yue and Lazaros Iliadis},
              series = {Communications in computer and information science},
                note = {Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012},
             address = {Heidelberg},
               title = {Engineering applications of neural networks},
           publisher = {Springer},
                year = {2012},
            keywords = {ARRAY(0x7f78592bbdc8)},
                 url = {http://eprints.lincoln.ac.uk/7434/},
            abstract = {Proceeedings of the 13th International Conference, EANN 2012, London, UK, September 20-23, 2012}
    }
  • C. Jayne, S. Yue, and L. Iliadis, “Engineering Applications of Neural Networks: 13th International Conference, EANN 2012 London, UK, September 20-23, 2012 Proceedings,” in 13th International Conference, EANN 2012, Chengdu, 2012.
    [BibTeX] [Abstract] [EPrints]

    .

    @inproceedings{lirolem11609,
              volume = {311},
               month = {September},
              author = {Chrisina Jayne and Shigang Yue and Lazaros Iliadis},
                note = { Conference Code:98083},
           booktitle = {13th International Conference, EANN 2012},
             address = {Chengdu},
               title = {Engineering Applications of Neural Networks: 13th International Conference, EANN 2012 London, UK, September 20-23, 2012 Proceedings},
           publisher = {Springer},
                year = {2012},
            keywords = {ARRAY(0x7f785945edf8)},
                 url = {http://eprints.lincoln.ac.uk/11609/},
            abstract = {.}
    }
  • C. Lang, S. Wachsmuth, M. Hanheide, and H. Wersing, “Facial communicative signals: valence recognition in task-oriented human-robot interaction,” International Journal of Social Robotics, vol. 4, iss. 3, pp. 249-262, 2012.
    [BibTeX] [Abstract] [EPrints]

    From the issue entitled "Measuring Human-Robots Interactions" This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates

    @article{lirolem6561,
              volume = {4},
              number = {3},
               month = {August},
              author = {Christian Lang and Sven Wachsmuth and Marc Hanheide and Heiko Wersing},
                note = {From the issue entitled "Measuring Human-Robots Interactions"
    This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates},
               title = {Facial communicative signals: valence recognition in task-oriented human-robot interaction},
           publisher = {Springer},
                year = {2012},
             journal = {International Journal of Social Robotics},
               pages = {249--262},
            keywords = {ARRAY(0x7f785944eaf8)},
                 url = {http://eprints.lincoln.ac.uk/6561/},
            abstract = {From the issue entitled "Measuring Human-Robots Interactions"
    This paper investigates facial communicative signals (head gestures, eye gaze, and facial expressions) as nonverbal feedback in human-robot interaction. Motivated by a discussion of the literature, we suggest scenario-specific investigations due to the complex nature of these signals and present an object-teaching scenario where subjects teach the names of objects to a robot, which in turn shall term these objects correctly afterwards. The robot?s verbal answers are to elicit facial communicative signals of its interaction partners. We investigated the human ability to recognize this spontaneous facial feedback and also the performance of two automatic recognition approaches. The first one is a static approach yielding baseline results, whereas the second considers the temporal dynamics and achieved classification rates}
    }
  • S. Liu, Y. Tang, C. Zhang, and S. Yue, “Self-map building in wireless sensor network based on TDOA measurements,” in IASTED International Conference on Artificial Intelligence and Soft Computing, Hamburg, 2012, pp. 150-155.
    [BibTeX] [Abstract] [EPrints]

    Node localization has long been established as a key problem in the sensor networks. Self-mapping in wireless sensor network which enables beacon-based systems to build a node map on-the-fly extends the range of the sensor network’s applications. A variety of self-mapping algorithms have been developed for the sensor networks. Some algorithms assume no information and estimate only the relative location of the sensor nodes. In this paper, we assume a very small percentage of the sensor nodes aware of their own locations, so the proposed algorithm estimates other node’s absolute location using the distance differences. In particular, time difference of arrival (TDOA) technology is adopted to obtain the distance difference. The obtained time difference accuracy is 10ns which corresponds to a distance difference error of 3m. We evaluate self-mapping’s accuracy with a small number of seed nodes. Overall, the accuracy and the coverage are shown to be comparable to those achieved results with other technologies and algorithms. Â\copyright 2012 IEEE.

    @inproceedings{lirolem10769,
               month = {September},
              author = {S. Liu and Y. Tang and C. Zhang and S. Yue},
                note = {Conference Code:94291},
           booktitle = {IASTED International Conference on Artificial Intelligence and Soft Computing},
             address = {Hamburg},
               title = {Self-map building in wireless sensor network based on TDOA measurements},
           publisher = {IASTED},
                year = {2012},
               pages = {150--155},
            keywords = {ARRAY(0x7f785940c208)},
                 url = {http://eprints.lincoln.ac.uk/10769/},
            abstract = {Node localization has long been established as a key problem in the sensor networks. Self-mapping in wireless sensor network which enables beacon-based systems to build a node map on-the-fly extends the range of the sensor network's applications. A variety of self-mapping algorithms have been developed for the sensor networks. Some algorithms assume no information and estimate only the relative location of the sensor nodes. In this paper, we assume a very small percentage of the sensor nodes aware of their own locations, so the proposed algorithm estimates other node's absolute location using the distance differences. In particular, time difference of arrival (TDOA) technology is adopted to obtain the distance difference. The obtained time difference accuracy is 10ns which corresponds to a distance difference error of 3m. We evaluate self-mapping's accuracy with a small number of seed nodes. Overall, the accuracy and the coverage are shown to be comparable to those achieved results with other technologies and algorithms. {\^A}{\copyright} 2012 IEEE.}
    }
  • M. Mangan and B. Webb, “Spontaneous formation of multiple routes in individual desert ants (Cataglyphis velox),” Behavioral Ecology, vol. 23, iss. 5, pp. 944-954, 2012.
    [BibTeX] [Abstract] [EPrints]

    Desert ants make use of various navigational techniques, including path integration and visual route following, to forage efficiently in their extremely hostile environment. Species-specific differences in navigation have been demonstrated, although it remains unknown if these divergences are caused by environmental adaptation. In this work, we report on the navigational strategies of the European ant Cataglyphis velox, which inhabits a visually cluttered environment similar to the Australian honey ant Melophorus bagoti, although it is more closely related to other North African Cataglyphis species. We show that C. velox learn visually guided routes, and these are individual to each forager. Routes can be recalled in the absence of global path integration information or when placed in conflict with this information. Individual C. velox foragers are also shown to learn multiple routes through their habitat. These routes are learned rapidly, stored in long-term memory, and recalled for guidance as appropriate. Desert ants have previously been shown to learn multiple routes in an experimental manipulation, but this is the first report of such behavior emerging spontaneously. Learning multiple paths through the habitat over successive journeys provides a mechanism by which ants could memorize a series of interlaced courses, and thus perform complex navigation, without necessarily having a map of the environment. Key words: Cataglyphis velox, desert ant, foraging, learning, navigation, route, visual navigation. [Behav Ecol]

    @article{lirolem23577,
              volume = {23},
              number = {5},
               month = {September},
              author = {Michael Mangan and Barbara Webb},
               title = {Spontaneous formation of multiple routes in individual desert ants (Cataglyphis velox)},
           publisher = {Oxford University Press for  International Society for Behavioral Ecology},
                year = {2012},
             journal = {Behavioral Ecology},
               pages = {944--954},
            keywords = {ARRAY(0x7f78592f0750)},
                 url = {http://eprints.lincoln.ac.uk/23577/},
            abstract = {Desert ants make use of various navigational techniques, including path integration and visual route following, to forage
    efficiently in their extremely hostile environment. Species-specific differences in navigation have been demonstrated, although
    it remains unknown if these divergences are caused by environmental adaptation. In this work, we report on the navigational
    strategies of the European ant Cataglyphis velox, which inhabits a visually cluttered environment similar to the Australian honey
    ant Melophorus bagoti, although it is more closely related to other North African Cataglyphis species. We show that C. velox learn
    visually guided routes, and these are individual to each forager. Routes can be recalled in the absence of global path integration
    information or when placed in conflict with this information. Individual C. velox foragers are also shown to learn multiple routes
    through their habitat. These routes are learned rapidly, stored in long-term memory, and recalled for guidance as appropriate.
    Desert ants have previously been shown to learn multiple routes in an experimental manipulation, but this is the first report
    of such behavior emerging spontaneously. Learning multiple paths through the habitat over successive journeys provides
    a mechanism by which ants could memorize a series of interlaced courses, and thus perform complex navigation, without
    necessarily having a map of the environment. Key words: Cataglyphis velox, desert ant, foraging, learning, navigation, route, visual
    navigation. [Behav Ecol]}
    }
  • O. Szymanezyk, T. Duckett, and P. Dickinson, “Agent-based crowd simulation in airports using games technology,” in Transactions on computational collective intelligence , SPRINGER, 2012.
    [BibTeX] [Abstract] [EPrints]

    We adapt popular video-games technology for an agent-based crowd simulation framework in an airport terminal. To achieve this, we investigate game technology, crowd simulation and the unique traits of airports. Our findings are implemented in a virtual airport environment that exploits a scalable layered intelligence technique in combination with physics middleware and a social force approach for crowd simulation. Our experiments show that the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating event triggered airport behaviour.

    @incollection{lirolem6574,
               month = {October},
              author = {Oliver Szymanezyk and Tom Duckett and Patrick Dickinson},
              series = {Lecture Notes in Computer Science},
                note = {Volume VIII, Issue 7430},
           booktitle = {Transactions on computational collective intelligence },
               title = {Agent-based crowd simulation in airports using games technology},
           publisher = {SPRINGER},
                year = {2012},
            keywords = {ARRAY(0x7f7859458ff0)},
                 url = {http://eprints.lincoln.ac.uk/6574/},
            abstract = {We adapt popular video-games technology for an agent-based crowd simulation framework in an airport terminal. To achieve this, we investigate game technology, crowd simulation and the unique traits of airports. Our findings are implemented in a virtual airport environment that exploits a scalable layered intelligence technique in combination with physics middleware and a social force approach for crowd simulation. Our experiments show that
    the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating event triggered airport behaviour.}
    }
  • Y. Tang, J. Peng, S. Yue, and J. Xu, “A primal dual proximal point method of Chambolle-Pock algorithms for ?1-TV minimization problems in image reconstruction,” in 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012, Chongqing, 2012, pp. 12-16.
    [BibTeX] [Abstract] [EPrints]

    Computed tomography (CT) image reconstruction problems can be solved by finding the minimizer of a suitable objective function. The objective function usually consists of a data fidelity term and a regularization term. Total variation (TV) minimization problems are widely used for solving incomplete data problems in CT image reconstruction. In this paper, we focus on the CT image reconstruction model which combines the TV regularization and ?1 data error term. We introduce a primal dual proximal point method of Chambolle-Pock algorithm to solve the proposed optimization problem. We tested it on computer simulated data and the experiment results shown it exhibited good performance when used to few-view CT image reconstruction. \copyright 2012 IEEE.

    @inproceedings{lirolem13409,
              author = {Y. Tang and Jigen Peng and Shigang Yue and Jiawei Xu},
           booktitle = {5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
             address = {Chongqing},
               title = {A primal dual proximal point method of Chambolle-Pock algorithms for ?1-TV minimization problems in image reconstruction},
           publisher = {IEEE},
             journal = {2012 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
               pages = {12--16},
                year = {2012},
            keywords = {ARRAY(0x7f78592c2d60)},
                 url = {http://eprints.lincoln.ac.uk/13409/},
            abstract = {Computed tomography (CT) image reconstruction problems can be solved by finding the minimizer of a suitable objective function. The objective function usually consists of a data fidelity term and a regularization term. Total variation (TV) minimization problems are widely used for solving incomplete data problems in CT image reconstruction. In this paper, we focus on the CT image reconstruction model which combines the TV regularization and ?1 data error term. We introduce a primal dual proximal point method of Chambolle-Pock algorithm to solve the proposed optimization problem. We tested it on computer simulated data and the experiment results shown it exhibited good performance when used to few-view CT image reconstruction. {\copyright} 2012 IEEE.}
    }
  • P. S. Teh, S. Yue, and A. B. J. Teoh, “Improving keystroke dynamics authentication system via multiple feature fusion scheme,” in 2012 International Conference on Cyber Security, Cyber Warfare and Digital Forensic (CyberSec), Kuala Lumpur, 2012, pp. 277-282.
    [BibTeX] [Abstract] [EPrints]

    This paper reports the performance and effect of diverse keystroke features combination on keystroke dynamic authentication system by using fusion scheme. First of all, four types of keystroke features are acquired from our collected dataset, later then transformed into similarity scores by the use of Gaussian Probability Density Function (GPD) and Direction Similarity Measure (DSM). Next, three fusion schemes are introduced to merge the scores pairing with six fusion rules. Result shows that the finest performance is obtained by the combination of both dwell time and flight time collectively. Finally, this experiment also investigates the effect of using larger dataset on performance, which turns out to be rather consistent. Â\copyright 2012 IEEE.

    @inproceedings{lirolem10860,
              author = {Pin Shen Teh and Shigang Yue and A. B. J. Teoh},
                note = {Conference Code:92830},
           booktitle = {2012 International Conference on Cyber Security, Cyber Warfare and Digital Forensic (CyberSec)},
             address = {Kuala Lumpur},
               title = {Improving keystroke dynamics authentication system via multiple feature fusion scheme},
           publisher = {IEEE},
               pages = {277--282},
                year = {2012},
            keywords = {ARRAY(0x7f785945a368)},
                 url = {http://eprints.lincoln.ac.uk/10860/},
            abstract = {This paper reports the performance and effect of diverse keystroke features combination on keystroke dynamic authentication system by using fusion scheme. First of all, four types of keystroke features are acquired from our collected dataset, later then transformed into similarity scores by the use of Gaussian Probability Density Function (GPD) and Direction Similarity Measure (DSM). Next, three fusion schemes are introduced to merge the scores pairing with six fusion rules. Result shows that the finest performance is obtained by the combination of both dwell time and flight time collectively. Finally, this experiment also investigates the effect of using larger dataset on performance, which turns out to be rather consistent. {\^A}{\copyright} 2012 IEEE.}
    }
  • Y. Utsumi, E. Sommerlade, N. Bellotto, and I. Reid, “Cognitive active vision for human identification,” in IEEE International Conference on Robotics and Automation (ICRA 2012), 2012.
    [BibTeX] [Abstract] [EPrints]

    We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.

    @inproceedings{lirolem4836,
           booktitle = {IEEE International Conference on Robotics and Automation (ICRA 2012)},
               month = {May},
               title = {Cognitive active vision for human identification},
              author = {Yuzuko Utsumi and Eric Sommerlade and Nicola Bellotto and Ian Reid},
                year = {2012},
                note = {We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph
    Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.},
            keywords = {ARRAY(0x7f78594021b8)},
                 url = {http://eprints.lincoln.ac.uk/4836/},
            abstract = {We describe an integrated, real-time multi-camera surveillance system that is able to find and track individuals, acquire and archive facial image sequences, and perform face recognition. The system is based around an inference engine that can extract high-level information from an observed scene, and generate appropriate commands for a set of pan-tilt-zoom (PTZ) cameras. The incorporation of a reliable facial recognition into the high-level feedback is a main novelty of our work, showing how high-level understanding of a scene can be used to deploy PTZ sensing resources effectively. The system comprises a distributed camera system using SQL tables as virtual communication channels, Situation Graph
    Trees for knowledge representation, inference and high-level camera control, and a variety of visual processing algorithms including an on-line acquisition of facial images, and on-line recognition of faces by comparing image sets using subspace distance. We provide an extensive evaluation of this method using our system for both acquisition of training data, and later recognition. A set of experiments in a surveillance scenario show the effectiveness of our approach and its potential for real applications of cognitive vision.}
    }
  • R. Wood, P. Baxter, and T. Belpaeme, “A Review of long-term memory in natural and synthetic systems,” Adaptive Behavior, vol. 20, iss. 2, pp. 81-103, 2012.
    [BibTeX] [Abstract] [EPrints]

    Memory may be broadly regarded as information gained from past experi- ence which is available in the service of ongoing and future adaptive behavior. The biological implementation ofmemory shares little with memory in synthetic cognitive systems where it is typically regarded as a passive storage structure. Neurophysiological evidence indicates that memory is neither passive nor cen- tralised. A review of the relevant literature in the biological and computer sciences is conducted and a novel methodology is applied that incorporates neuroethological approaches with general biological inspiration in the design of synthetic cognitive systems: a case study regarding episodic memory provides an illustration of the utility of this methodology. As a consequence of applying this approach to the reinterpretation of the implementation of memory in syn- thetic systems, four fundamental functional principles are derived that are in accordance with neuroscientific theory, and which may be applied to the design of more adaptive and robust synthetic cognitive systems: priming, cross-modal associations, cross-modal coordination without semantic information transfer, and global system behavior resulting from activation dynamics within the mem- ory system.

    @article{lirolem23079,
              volume = {20},
              number = {2},
               month = {April},
              author = {Rachel Wood and Paul Baxter and Tony Belpaeme},
               title = {A Review of long-term memory in natural and synthetic systems},
           publisher = {Sage for International Society for Adaptive Behavior (ISAB)},
                year = {2012},
             journal = {Adaptive Behavior},
               pages = {81--103},
            keywords = {ARRAY(0x7f78590bfa30)},
                 url = {http://eprints.lincoln.ac.uk/23079/},
            abstract = {Memory may be broadly regarded as information gained from past experi- ence which is available in the service of ongoing and future adaptive behavior. The biological implementation ofmemory shares little with memory in synthetic cognitive systems where it is typically regarded as a passive storage structure. Neurophysiological evidence indicates that memory is neither passive nor cen- tralised. A review of the relevant literature in the biological and computer sciences is conducted and a novel methodology is applied that incorporates neuroethological approaches with general biological inspiration in the design of synthetic cognitive systems: a case study regarding episodic memory provides an illustration of the utility of this methodology. As a consequence of applying this approach to the reinterpretation of the implementation of memory in syn- thetic systems, four fundamental functional principles are derived that are in accordance with neuroscientific theory, and which may be applied to the design of more adaptive and robust synthetic cognitive systems: priming, cross-modal associations, cross-modal coordination without semantic information transfer, and global system behavior resulting from activation dynamics within the mem- ory system.}
    }
  • J. Xu and S. Yue, “Visual based contour detection by using the improved short path finding,” Communications in Computer and Information Science, vol. 311, pp. 145-151, 2012.
    [BibTeX] [Abstract] [EPrints]

    Contour detection is an important characteristic of human vision perception. Humans can easily find the objects contour in a complex visual scene; however, traditional computer vision cannot do well. This paper primarily concerned with how to track the objects contour using a human-like vision. In this article, we propose a biologically motivated computational model to track and detect the objects contour. Even the previous research has proposed some models by using the Dijkstra algorithm 1, our work is to mimic the human eye movement and imitate saccades in our humans. We use natural images with associated ground truth contour maps to assess the performance of the proposed operator regarding the detection of contours while suppressing texture edges. The results show that our method enhances contour detection in cluttered visual scenes more effectively than classical edge detectors proposed by other methods. Â\copyright Springer-Verlag Berlin Heidelberg 2012.

    @article{lirolem11608,
              volume = {311},
               month = {September},
              author = {Jiawei Xu and Shigang Yue},
                note = {13th International Conference, EANN 2012, London, UK, September 20-23, 2012. Conference Code:98083},
             address = {Chengdu},
               title = {Visual based contour detection by using the improved short path finding},
           publisher = {Springer Verlag},
                year = {2012},
             journal = {Communications in Computer and Information Science},
               pages = {145--151},
            keywords = {ARRAY(0x7f7858f49308)},
                 url = {http://eprints.lincoln.ac.uk/11608/},
            abstract = {Contour detection is an important characteristic of human vision perception. Humans can easily find the objects contour in a complex visual scene; however, traditional computer vision cannot do well. This paper primarily concerned with how to track the objects contour using a human-like vision. In this article, we propose a biologically motivated computational model to track and detect the objects contour. Even the previous research has proposed some models by using the Dijkstra algorithm 1, our work is to mimic the human eye movement and imitate saccades in our humans. We use natural images with associated ground truth contour maps to assess the performance of the proposed operator regarding the detection of contours while suppressing texture edges. The results show that our method enhances contour detection in cluttered visual scenes more effectively than classical edge detectors proposed by other methods. {\^A}{\copyright} Springer-Verlag Berlin Heidelberg 2012.}
    }
  • J. Xu and S. Yue, “A top-down attention model based on the semi-supervised learning,” in 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012, Chongqing, 2012, pp. 1011-1014.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we proposed a top-down motion tracking model to detect the attention region. Many biological inspired systems have been studied and most of them are consisted by bottom-up mechanisms and top-down processes. Top-down attention is guided by task-driven information that is acquired through learning procedures. Our model improves the top-down mechanisms by using a probability map (PM). The PM follows to track if all the potential locations of targets based on the information contained in the frame sequences. By using this, PM can be regarded as a short term memory for attended saliency regions. This function is similar to the dorsal stream of V1 primary area. The semi-learning model constructs an efficient mechanism for attention detection to simulate the eye movements and fixations in our human visual systems. Generally, our work is to mimic human visual systems and it will further be applied on the robotics platform. From the random selected video clips, our performances are better than other state-of-the-art approaches. Â\copyright 2012 IEEE.

    @inproceedings{lirolem13408,
              author = {Jiawei Xu and Shigang Yue},
           booktitle = {5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
             address = {Chongqing},
               title = {A top-down attention model based on the semi-supervised learning},
           publisher = {IEEE},
             journal = {2012 5th International Conference on Biomedical Engineering and Informatics, BMEI 2012},
               pages = {1011--1014},
                year = {2012},
            keywords = {ARRAY(0x7f7859435a70)},
                 url = {http://eprints.lincoln.ac.uk/13408/},
            abstract = {In this paper, we proposed a top-down motion tracking model to detect the attention region. Many biological inspired systems have been studied and most of them are consisted by bottom-up mechanisms and top-down processes. Top-down attention is guided by task-driven information that is acquired through learning procedures. Our model improves the top-down mechanisms by using a probability map (PM). The PM follows to track if all the potential locations of targets based on the information contained in the frame sequences. By using this, PM can be regarded as a short term memory for attended saliency regions. This function is similar to the dorsal stream of V1 primary area. The semi-learning model constructs an efficient mechanism for attention detection to simulate the eye movements and fixations in our human visual systems. Generally, our work is to mimic human visual systems and it will further be applied on the robotics platform. From the random selected video clips, our performances are better than other state-of-the-art approaches. {\^A}{\copyright} 2012 IEEE.}
    }
  • S. Yue and C. Rind, “Visually stimulated motor control for a robot with a pair of LGMD visual neural networks,” International Journal of Advanced Mechatronic Systems, vol. 4, iss. 5/6, pp. 237-247, 2012.
    [BibTeX] [Abstract] [EPrints]

    In this paper, we proposed a visually stimulated motor control (VSMC) system for autonomous navigation of mobile robots. Inspired from a locusts? motion sensitive interneuron ? lobula giant movement detector (LGMD), the presented VSMC system enables a robot exploring local paths or interacting with dynamic objects effectively using visual input only. The VSMC consists of a pair of LGMD visual neural networks and a simple motor command generator. Each LGMD processes images covering part of the wide field of view and extracts relevant visual cues. The outputs from the two LGMDs are compared and interpreted into executable motor commands directly. These motor commands are then executed by the robot?s wheel control system in real-time to generate corresponded motion adjustment accordingly. Our experiments showed that this bio-inspired VSMC system worked well in different scenarios.

    @article{lirolem9309,
              volume = {4},
              number = {5/6},
               month = {October},
              author = {Shigang Yue and Claire Rind},
                note = {Special Issue on Advanced Application of Modelling, Identification and Control},
               title = {Visually stimulated motor control for a robot with a pair of LGMD visual neural networks},
           publisher = {Inderscience},
                year = {2012},
             journal = {International Journal of Advanced Mechatronic Systems},
               pages = {237--247},
            keywords = {ARRAY(0x7f785940b998)},
                 url = {http://eprints.lincoln.ac.uk/9309/},
            abstract = {In this paper, we proposed a visually stimulated motor control (VSMC) system
    for autonomous navigation of mobile robots. Inspired from a locusts? motion sensitive
    interneuron ? lobula giant movement detector (LGMD), the presented VSMC system enables a
    robot exploring local paths or interacting with dynamic objects effectively using visual input
    only. The VSMC consists of a pair of LGMD visual neural networks and a simple motor
    command generator. Each LGMD processes images covering part of the wide field of view and
    extracts relevant visual cues. The outputs from the two LGMDs are compared and interpreted
    into executable motor commands directly. These motor commands are then executed by the
    robot?s wheel control system in real-time to generate corresponded motion adjustment
    accordingly. Our experiments showed that this bio-inspired VSMC system worked well in
    different scenarios.}
    }

2011

  • F. Arvin, K. Samsudin, A. R. Ramli, and M. Bekravi, “Imitation of honeybee aggregation with collective behavior of swarm robots,” International Journal of Computational Intelligence Systems, vol. 4, iss. 5, pp. 739-748, 2011.
    [BibTeX] [Abstract] [EPrints]

    This paper analyzes the collective behaviors of swarm robots that play role in the aggregation scenario. Honeybee aggregation is an inspired behavior of young honeybees which tend to aggregate around an optimal zone. This aggregation is implemented based on variation of parameters values. In the second phase, two modifications on original honeybee aggregation namely dynamic velocity and comparative waiting time are proposed. Results of the performed experiments showed the significant differences in collective behavior of the swarm system for different algorithms.

    @article{lirolem5515,
              volume = {4},
              number = {5},
               month = {August},
              author = {Farshad Arvin and Khairulmizam Samsudin and Abdul Rahman Ramli  and Masoud Bekravi},
               title = {Imitation of honeybee aggregation with collective behavior of swarm robots},
           publisher = {Taylor \& Francis},
                year = {2011},
             journal = {International Journal of Computational Intelligence Systems},
               pages = {739 --748},
            keywords = {ARRAY(0x7f7859410b40)},
                 url = {http://eprints.lincoln.ac.uk/5515/},
            abstract = {This paper analyzes the collective behaviors of swarm robots that play role in the aggregation scenario. Honeybee
    aggregation is an inspired behavior of young honeybees which tend to aggregate around an optimal zone. This
    aggregation is implemented based on variation of parameters values. In the second phase, two modifications on original honeybee aggregation namely dynamic velocity and comparative waiting time are proposed. Results of the performed experiments showed the significant differences in collective behavior of the swarm system for different algorithms.}
    }
  • F. Arvin, S. Doraisamy, K. Samsudin, F. A. Ahmad, and A. R. Ramli, “Implementation of a cue-based aggregation with a swarm robotic system,” in Third Knowledge Technology Week, KTW 2011, 2011, pp. 113-122.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an aggregation behavior using a robot swarm. Swarm robotics takes inspiration from behaviors of social insects. BEECLUST is an aggregation control that is inspired from thermotactic behavior of young honeybees in producing clusters. In this study, aggregation method is implemented with a modification on original BEECLUST. Both aggregations are performed using real and simulated robots. We aim to demonstrate that, a simple change in control of individual robots results in significant changes in collective behavior of the swarm. In addition, the behavior of the swarm is modeled by a macroscopic modeling based on a probability control. The presented model in this study could depict the behavior of swarm throughout the performed scenarios with real and simulated robots.

    @inproceedings{lirolem6086,
              volume = {295},
              number = {2},
               month = {July},
              author = {Farshad Arvin and Shyamala Doraisamy and Khairulmizam Samsudin and Faisul Arif Ahmad and Abdul Rahman  Ramli},
                note = {Third Knowledge Technology Week, KTW 2011, Kajang, Malaysia, July 18-22, 2011. Revised Selected Papers},
           booktitle = {Third Knowledge Technology Week, KTW 2011},
               title = {Implementation of a cue-based aggregation with a swarm robotic system},
           publisher = {Springer},
                year = {2011},
               pages = {113--122},
            keywords = {ARRAY(0x7f7858f64768)},
                 url = {http://eprints.lincoln.ac.uk/6086/},
            abstract = {This paper presents an aggregation behavior using a robot swarm. Swarm robotics takes inspiration from behaviors of social insects. BEECLUST is an aggregation control that is inspired from thermotactic behavior of young honeybees in producing clusters. In this study, aggregation method is implemented with a modification on original BEECLUST. Both aggregations are performed using real and simulated robots. We aim to demonstrate that, a simple change in control of individual robots results in significant changes in collective behavior of the swarm. In addition, the behavior of the swarm is modeled by a macroscopic modeling based on a probability control. The presented model in this study could depict the behavior of swarm throughout the performed scenarios with real and simulated robots.}
    }
  • F. Arvin, S. Doraisamy, and E. S. Khorasani, “Frequency shifting approach towards textual transcription of heartbeat sounds,” Biological Procedures Online, vol. 13, iss. 7, pp. 1-7, 2011.
    [BibTeX] [Abstract] [EPrints]

    Auscultation is an approach for diagnosing many cardiovascular problems. Automatic analysis of heartbeat sounds and extraction of its audio features can assist physicians towards diagnosing diseases. Textual transcription allows recording a continuous heart sound stream using a text format which can be stored in very small memory in comparison with other audio formats. In addition, a text-based data allows applying indexing and searching techniques to access to the critical events. Hence, the transcribed heartbeat sounds provides useful information to monitor the behavior of a patient for the long duration of time. This paper proposes a frequency shifting method in order to improve the performance of the transcription. The main objective of this study is to transfer the heartbeat sounds to the music domain. The proposed technique is tested with 100 samples which were recorded from different heart diseases categories. The observed results show that, the proposed shifting method significantly improves the performance of the transcription.

    @article{lirolem5794,
              volume = {13},
              number = {7},
               month = {November},
              author = {Farshad Arvin and Shyamala Doraisamy and Ehsan Safar Khorasani},
               title = {Frequency shifting approach towards textual transcription of heartbeat sounds},
           publisher = {Springer},
                year = {2011},
             journal = {Biological Procedures Online},
               pages = {1--7},
            keywords = {ARRAY(0x7f78593e2b50)},
                 url = {http://eprints.lincoln.ac.uk/5794/},
            abstract = {Auscultation is an approach for diagnosing many cardiovascular problems. Automatic analysis of heartbeat sounds and extraction of its audio features can assist physicians towards diagnosing diseases. Textual transcription allows recording a continuous heart sound stream using a text format which can be stored in very small memory in comparison with other audio formats. In addition, a text-based data allows applying indexing and searching techniques to access to the critical events. Hence, the transcribed heartbeat sounds provides useful information to monitor the behavior of a patient for the long duration of time. This paper proposes a frequency shifting method in order to improve the performance of the transcription. The main objective of this study is to transfer the heartbeat sounds to the music domain. The proposed technique is tested with 100 samples which were recorded from different heart diseases categories. The observed results show that, the proposed shifting method significantly improves the performance of the transcription.}
    }
  • R. S. Aylett, G. Castellano, B. Raducanu, A. Paiva, and M. Hanheide, “Long-term socially perceptive and interactive robot companions: challenges and future perspective,” in Conference of 2011 ACM International Conference on Multimodal Interaction, ICMI’11, Alicante, 2011, pp. 323-326.
    [BibTeX] [Abstract] [EPrints]

    This paper gives a brief overview of the challenges for multi-model perception and generation applied to robot companions located in human social environments. It reviews the current position in both perception and generation and the immediate technical challenges and goes on to consider the extra issues raised by embodiment and social context. Finally, it briefly discusses the impact of systems that must function continually over months rather than just for a few hours. Â\copyright 2011 ACM.

    @inproceedings{lirolem8314,
               month = {November},
              author = {Ruth S. Aylett and Ginevra Castellano and Bogdan Raducanu and Ana Paiva and Marc Hanheide},
                note = {Conference Code: 87685},
           booktitle = {Conference of 2011 ACM International Conference on Multimodal Interaction, ICMI'11},
               title = {Long-term socially perceptive and interactive robot companions: challenges and future perspective},
             address = {Alicante},
           publisher = {ACM},
                year = {2011},
             journal = {ICMI'11 - Proceedings of the 2011 ACM International Conference on Multimodal Interaction},
               pages = {323--326},
            keywords = {ARRAY(0x7f78590b0e88)},
                 url = {http://eprints.lincoln.ac.uk/8314/},
            abstract = {This paper gives a brief overview of the challenges for multi-model perception and generation applied to robot companions located in human social environments. It reviews the current position in both perception and generation and the immediate technical challenges and goes on to consider the extra issues raised by embodiment and social context. Finally, it briefly discusses the impact of systems that must function continually over months rather than just for a few hours. {\^A}{\copyright} 2011 ACM.}
    }
  • V. Belevskiy and S. Yue, “Near range pedestrian collision detection using bio-inspired visual neural networks,” in 2011 Seventh International Conference on Natural Computation, 2011, pp. 786-790.
    [BibTeX] [Abstract] [EPrints]

    New vehicular safety standards require the development of pedestrian collision detection systems that can trigger the deployment of active impact alleviation measures from the vehicle prior to a collision. In this paper, we propose a new vision-based system for near-range pedestrian collision detection. The low-level system uses a bio-inspired visual neural network, which emulates the visual system of the locust, to detect visual cues relevant to objects in front of a moving car. At a higher level, the system employs a neural-network classifier to identify dangerous pedestrian positions, triggering an alarm signal. The system was tuned via simulation and tested using recorded video sequences of real vehicle impacts. The experiment results demonstrate that the system is able to discriminate between pedestrians in dangerous and safe positions, triggering alarms accordingly.

    @inproceedings{lirolem12818,
           booktitle = {2011 Seventh International Conference on Natural Computation},
               month = {July},
               title = {Near range pedestrian collision detection using bio-inspired visual neural networks},
              author = {Vladimir Belevskiy and Shigang Yue},
           publisher = {IEEE},
                year = {2011},
               pages = {786--790},
            keywords = {ARRAY(0x7f78594253f0)},
                 url = {http://eprints.lincoln.ac.uk/12818/},
            abstract = {New vehicular safety standards require the development of pedestrian collision detection systems that can trigger the deployment of active impact alleviation measures from the vehicle prior to a collision. In this paper, we propose a new vision-based system for near-range pedestrian collision detection. The low-level system uses a bio-inspired visual neural network, which emulates the visual system of the locust, to detect visual cues relevant to objects in front of a moving car. At a higher level, the system employs a neural-network classifier to identify dangerous pedestrian positions, triggering an alarm signal. The system was tuned via simulation and tested using recorded video sequences of real vehicle impacts. The experiment results demonstrate that the system is able to discriminate between pedestrians in dangerous and safe positions, triggering alarms accordingly.}
    }
  • H. Cuayahuitl, “Spatially-aware dialogue control using hierarchical reinforcement learning,” ACM Transactions on Speech and Language Processing (TSLP), vol. 7, iss. 3, 2011.
    [BibTeX] [Abstract] [EPrints]

    This article addresses the problem of scalable optimization for spatially-aware dialogue systems. These kinds of systems must perceive, reason, and act about the spatial environment where they are embedded. We formulate the problem in terms of Semi-Markov Decision Processes and propose a hierarchical reinforcement learning approach to optimize subbehaviors rather than full behaviors. Because of the vast number of policies that are required to control the interaction in a dynamic environment (e.g., a dialogue system assisting a user to navigate in a building from one location to another), our learning approach is based on two stages: (a) the first stage learns low-level behavior, in advance; and (b) the second stage learns high-level behavior, in real time. For such a purpose we extend an existing algorithm in the literature of reinforcement learning in order to support reusable policies and therefore to perform fast learning. We argue that our learning approach makes the problem feasible, and we report on a novel reinforcement learning dialogue system that performs a joint optimization between dialogue and spatial behaviors. Our experiments, using simulated and real environments, are based on a text-based dialogue system for indoor navigation. Experimental results in a realistic environment reported an overall user satisfaction result of 89\%, which suggests that our proposed approach is attractive for its application in real interactions as it combines fast learning with adaptive and reasonable behavior.

    @article{lirolem22209,
              volume = {7},
              number = {3},
               month = {May},
              author = {Heriberto Cuayahuitl},
               title = {Spatially-aware dialogue control using hierarchical reinforcement learning},
           publisher = {Association for Computing Machinery},
             journal = {ACM Transactions on Speech and Language Processing (TSLP)},
                year = {2011},
            keywords = {ARRAY(0x7f7859466ab0)},
                 url = {http://eprints.lincoln.ac.uk/22209/},
            abstract = {This article addresses the problem of scalable optimization for spatially-aware dialogue systems. These kinds of systems must perceive, reason, and act about the spatial environment where they are embedded. We formulate the problem in terms of Semi-Markov Decision Processes and propose a hierarchical reinforcement learning approach to optimize subbehaviors rather than full behaviors. Because of the vast number of policies that are required to control the interaction in a dynamic environment (e.g., a dialogue system assisting a user to navigate in a building from one location to another), our learning approach is based on two stages: (a) the first stage learns low-level behavior, in advance; and (b) the second stage learns high-level behavior, in real time. For such a purpose we extend an existing algorithm in the literature of reinforcement learning in order to support reusable policies and therefore to perform fast learning. We argue that our learning approach makes the problem feasible, and we report on a novel reinforcement learning dialogue system that performs a joint optimization between dialogue and spatial behaviors. Our experiments, using simulated and real environments, are based on a text-based dialogue system for indoor navigation. Experimental results in a realistic environment reported an overall user satisfaction result of 89\%, which suggests that our proposed approach is attractive for its application in real interactions as it combines fast learning with adaptive and reasonable behavior.}
    }
  • F. Dayoub, G. Cielniak, and T. Duckett, “Long-term experiments with an adaptive spherical view representation for navigation in changing environments,” Robotics and Autonomous Systems, vol. 59, iss. 5, pp. 285-295, 2011.
    [BibTeX] [Abstract] [EPrints]

    Real-world environments such as houses and offices change over time, meaning that a mobile robot?s map will become out of date. In this work, we introduce a method to update the reference views in a hybrid metric-topological map so that a mobile robot can continue to localize itself in a changing environment. The updating mechanism, based on the multi-store model of human memory, incorporates a spherical metric representation of the observed visual features for each node in the map, which enables the robot to estimate its heading and navigate using multi-view geometry, as well as representing the local 3D geometry of the environment. A series of experiments demonstrate the persistence performance of the proposed system in real changing environments, including analysis of the long-term stability.

    @article{lirolem6046,
              volume = {59},
              number = {5},
               month = {May},
              author = {Feras Dayoub and Grzegorz Cielniak and Tom Duckett},
               title = {Long-term experiments with an adaptive spherical view representation for navigation in changing environments},
           publisher = {Elsevier},
                year = {2011},
             journal = {Robotics and Autonomous Systems},
               pages = {285--295},
            keywords = {ARRAY(0x7f7858f6eb00)},
                 url = {http://eprints.lincoln.ac.uk/6046/},
            abstract = {Real-world environments such as houses and offices change over time, meaning that a mobile robot?s map will become out of date. In this work, we introduce a method to update the reference views in a hybrid metric-topological map so that a mobile robot can continue to localize itself in a changing environment. The updating mechanism, based on the multi-store model of human memory, incorporates a spherical metric representation of the observed visual features for each node in the map, which enables the robot to estimate its heading and navigate using multi-view geometry, as well as representing the local 3D geometry of the environment. A series of experiments demonstrate the persistence performance of the proposed system in real changing environments, including analysis of the long-term stability.}
    }
  • F. Dayoub, G. Cielniak, and T. Duckett, “Long-term experiment using an adaptive appearance-based map for visual navigation by mobile robots,” in Towards autonomous robotic systems, Sheffield: Springer, 2011, vol. 6856 , pp. 400-401.
    [BibTeX] [Abstract] [EPrints]

    Building functional and useful mobile service robots means that these robots have to be able to share physical spaces with humans, and to update their internal representation of the world in response to changes in the arrangement of objects and appearance of the environment – changes that may be spontaneous and unpredictable – as a result of human activities. However, almost all past research on robot mapping addresses only the initial learning of an environment, a phase which will only be a short moment in the lifetime of a service robot that may be expected to operate for many years. \copyright 2011 Springer-Verlag Berlin Heidelberg.

    @incollection{lirolem10338,
              volume = {6856 },
               month = {August},
              author = {Feras Dayoub and Grzegorz Cielniak and Tom Duckett},
              series = {Lecture Notes in Computer Science},
                note = {12th Annual Conference, TAROS 2011, Sheffield, UK, August 31 ? September 2, 2011. Proceedings},
           booktitle = {Towards autonomous robotic systems},
               title = {Long-term experiment using an adaptive appearance-based map for visual navigation by mobile robots},
             address = {Sheffield},
           publisher = {Springer},
                year = {2011},
               pages = {400--401},
            keywords = {ARRAY(0x7f785940c4f0)},
                 url = {http://eprints.lincoln.ac.uk/10338/},
            abstract = {Building functional and useful mobile service robots means that these robots have to be able to share physical spaces with humans, and to update their internal representation of the world in response to changes in the arrangement of objects and appearance of the environment - changes that may be spontaneous and unpredictable - as a result of human activities. However, almost all past research on robot mapping addresses only the initial learning of an environment, a phase which will only be a short moment in the lifetime of a service robot that may be expected to operate for many years. {\copyright} 2011 Springer-Verlag Berlin Heidelberg.}
    }
  • R. Golombek, S. Wrede, M. Hanheide, and M. Heckmann, “Online data-driven fault detection for robotic systems,” in Conference of 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems: Celebrating 50 Years of Robotics, IROS’11, San Francisco, CA, 2011, pp. 3011-3016.
    [BibTeX] [Abstract] [EPrints]

    In this paper we demonstrate the online applicability of the fault detection and diagnosis approach which we previously developed and published in 1. In our former work we showed that a purely data driven fault detection approach can be successfully built based on monitored inter-component communication data of a robotic system and used for a-posteriori fault detection. Here we propose an extension to this approach which is capable of online learning of the fault model as well as for online fault detection. We evaluate the application of our approach in the context of a RoboCup task executed by our service robot BIRON in corporation with an expert user. Â\copyright 2011 IEEE.

    @inproceedings{lirolem8313,
               month = {September},
              author = {R. Golombek and S. Wrede and Marc Hanheide and M. Heckmann},
                note = {Conference Code: 87712},
           booktitle = {Conference of 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems: Celebrating 50 Years of Robotics, IROS'11},
               title = {Online data-driven fault detection for robotic systems},
             address = {San Francisco, CA},
           publisher = {IEEE},
                year = {2011},
             journal = {IEEE International Conference on Intelligent Robots and Systems},
               pages = {3011--3016},
            keywords = {ARRAY(0x7f785945f0e0)},
                 url = {http://eprints.lincoln.ac.uk/8313/},
            abstract = {In this paper we demonstrate the online applicability of the fault detection and diagnosis approach which we previously developed and published in 1. In our former work we showed that a purely data driven fault detection approach can be successfully built based on monitored inter-component communication data of a robotic system and used for a-posteriori fault detection. Here we propose an extension to this approach which is capable of online learning of the fault model as well as for online fault detection. We evaluate the application of our approach in the context of a RoboCup task executed by our service robot BIRON in corporation with an expert user. {\^A}{\copyright} 2011 IEEE.}
    }
  • M. Hanheide, C. Gretton, R. W. Dearden, N. A. Hawes, J. L. Wyatt, M. Goedelbecker, A. Pronobis, A. Aydemir, and H. Zender, “Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour,” in Twenty-Second International Joint Conference on Artificial Intelligence, 2011, pp. 2442-2449.
    [BibTeX] [Abstract] [EPrints]

    Robots must perform tasks efficiently and reli- ably while acting under uncertainty. One way to achieve efficiency is to give the robot common- sense knowledge about the structure of the world. Reliable robot behaviour can be achieved by mod- elling the uncertainty in the world probabilistically. We present a robot system that combines these two approaches and demonstrate the improvements in efficiency and reliability that result. Our first con- tribution is a probabilistic relational model integrat- ing common-sense knowledge about the world in general, with observations of a particular environ- ment. Our second contribution is a continual plan- ning system which is able to plan in the large prob- lems posed by that model, by automatically switch- ing between decision-theoretic and classical proce- dures. We evaluate our system on object search tasks in two different real-world indoor environ- ments. By reasoning about the trade-offs between possible courses of action with different informa- tional effects, and exploiting the cues and general structures of those environments, our robot is able to consistently demonstrate efficient and reliable goal-directed behaviour.

    @inproceedings{lirolem6756,
               month = {July},
              author = {Marc Hanheide and Charles Gretton and Richard W. Dearden and Nick A. Hawes and Jeremy L. Wyatt and Moritz Goedelbecker and Andrzej Pronobis and Alper Aydemir and Hendrik Zender},
                note = {Robots must perform tasks efficiently and reli- ably while acting under uncertainty. One way to achieve efficiency is to give the robot common- sense knowledge about the structure of the world. Reliable robot behaviour can be achieved by mod- elling the uncertainty in the world probabilistically. We present a robot system that combines these two approaches and demonstrate the improvements in efficiency and reliability that result. Our first con- tribution is a probabilistic relational model integrat- ing common-sense knowledge about the world in general, with observations of a particular environ- ment. Our second contribution is a continual plan- ning system which is able to plan in the large prob- lems posed by that model, by automatically switch- ing between decision-theoretic and classical proce- dures. We evaluate our system on object search tasks in two different real-world indoor environ- ments. By reasoning about the trade-offs between possible courses of action with different informa- tional effects, and exploiting the cues and general structures of those environments, our robot is able to consistently demonstrate efficient and reliable goal-directed behaviour.},
           booktitle = {Twenty-Second International Joint Conference on Artificial Intelligence},
              editor = {B. Gottfried and H. Aghajan},
               title = {Exploiting probabilistic knowledge under uncertain sensing for efficient robot behaviour},
           publisher = {International Joint Conferences on Artiicial Intelligence},
                year = {2011},
               pages = {2442--2449},
            keywords = {ARRAY(0x7f78590bb020)},
                 url = {http://eprints.lincoln.ac.uk/6756/},
            abstract = {Robots must perform tasks efficiently and reli- ably while acting under uncertainty. One way to achieve efficiency is to give the robot common- sense knowledge about the structure of the world. Reliable robot behaviour can be achieved by mod- elling the uncertainty in the world probabilistically. We present a robot system that combines these two approaches and demonstrate the improvements in efficiency and reliability that result. Our first con- tribution is a probabilistic relational model integrat- ing common-sense knowledge about the world in general, with observations of a particular environ- ment. Our second contribution is a continual plan- ning system which is able to plan in the large prob- lems posed by that model, by automatically switch- ing between decision-theoretic and classical proce- dures. We evaluate our system on object search tasks in two different real-world indoor environ- ments. By reasoning about the trade-offs between possible courses of action with different informa- tional effects, and exploiting the cues and general structures of those environments, our robot is able to consistently demonstrate efficient and reliable goal-directed behaviour.}
    }
  • H. Hauser, G. Neumann, A. J. Ijspeert, and W. Maass, “Biologically inspired kinematic synergies enable linear balance control of a humanoid robot,” Biological Cybernetics, vol. 104, iss. 4-5, pp. 235-249, 2011.
    [BibTeX] [Abstract] [EPrints]

    Despite many efforts, balance control of humanoid robots in the presence of unforeseen external or internal forces has remained an unsolved problem. The difficulty of this problem is a consequence of the high dimensionality of the action space of a humanoid robot, due to its large number of degrees of freedom (joints), and of non-linearities in its kinematic chains. Biped biological organisms face similar difficulties, but have nevertheless solved this problem. Experimental data reveal that many biological organisms reduce the high dimensionality of their action space by generating movements through linear superposition of a rather small number of stereotypical combinations of simultaneous movements of many joints, to which we refer as kinematic synergies in this paper. We show that by constructing two suitable non-linear kinematic synergies for the lower part of the body of a humanoid robot, balance control can in fact be reduced to a linear control problem, at least in the case of relatively slow movements. We demonstrate for a variety of tasks that the humanoid robot HOAP-2 acquires through this approach the capability to balance dynamically against unforeseen disturbances that may arise from external forces or from manipulating unknown loads.

    @article{lirolem25794,
              volume = {104},
              number = {4-5},
               month = {May},
              author = {Helmut Hauser and Gerhard Neumann and Auke J. Ijspeert and Wolfgang Maass},
               title = {Biologically inspired kinematic synergies enable linear balance control of a humanoid robot},
           publisher = {Springer},
                year = {2011},
             journal = {Biological Cybernetics},
               pages = {235--249},
            keywords = {ARRAY(0x7f7858f49338)},
                 url = {http://eprints.lincoln.ac.uk/25794/},
            abstract = {Despite many efforts, balance control of humanoid robots in the presence of unforeseen external or internal forces has remained an unsolved problem. The difficulty of this problem is a consequence of the high dimensionality of the action space of a humanoid robot, due to its large number of degrees of freedom (joints), and of non-linearities in its kinematic chains. Biped biological organisms face similar difficulties, but have nevertheless solved this problem. Experimental data reveal that many biological organisms reduce the high dimensionality of their action space by generating movements through linear superposition of a rather small number of stereotypical combinations of simultaneous movements of many joints, to which we refer as kinematic synergies in this paper. We show that by constructing two suitable non-linear kinematic synergies for the lower part of the body of a humanoid robot, balance control can in fact be reduced to a linear control problem, at least in the case of relatively slow movements. We demonstrate for a variety of tasks that the humanoid robot HOAP-2 acquires through this approach the capability to balance dynamically against unforeseen disturbances that may arise from external forces or from manipulating unknown loads.}
    }
  • N. Hawes, M. Hanheide, J. Hargreaves, B. Page, H. Zender, and P. Jensfelt, “Home alone: autonomous extension and correction of spatial representations,” in 2011 IEEE International Conference on Robotics and Automation (ICRA), 2011, pp. 3907-3914.
    [BibTeX] [Abstract] [EPrints]

    In this paper we present an account of the problems faced by a mobile robot given an incomplete tour of an unknown environment, and introduce a collection of techniques which can generate successful behaviour even in the presence of such problems. Underlying our approach is the principle that an autonomous system must be motivated to act to gather new knowledge, and to validate and correct existing knowledge. This principle is embodied in Dora, a mobile robot which features the aforementioned techniques: shared representations, non-monotonic reasoning, and goal generation and management. To demonstrate how well this collection of techniques work in real-world situations we present a comprehensive analysis of the Dora system?s performance over multiple tours in an indoor environment. In this analysis Dora successfully completed 18 of 21 attempted runs, with all but 3 of these successes requiring one or more of the integrated techniques to recover from problems.

    @inproceedings{lirolem6764,
               month = {May},
              author = {Nick Hawes and Marc Hanheide and Jack Hargreaves and Ben Page and Hendrik Zender and Patric Jensfelt},
                note = {In this paper we present an account
    of the problems faced by a mobile robot given
    an incomplete tour of an unknown environment,
    and introduce a collection of techniques which can
    generate successful behaviour even in the presence
    of such problems. Underlying our approach is the
    principle that an autonomous system must be motivated
    to act to gather new knowledge, and to validate
    and correct existing knowledge. This principle is
    embodied in Dora, a mobile robot which features
    the aforementioned techniques: shared representations,
    non-monotonic reasoning, and goal generation
    and management. To demonstrate how well this
    collection of techniques work in real-world situations
    we present a comprehensive analysis of the Dora
    system?s performance over multiple tours in an indoor
    environment. In this analysis Dora successfully
    completed 18 of 21 attempted runs, with all but
    3 of these successes requiring one or more of the
    integrated techniques to recover from problems.},
           booktitle = {2011 IEEE International Conference on Robotics and Automation (ICRA)},
              editor = {B. Gottfried and H. Aghajan},
               title = {Home alone: autonomous extension and correction of spatial
    representations},
           publisher = {IEEE},
                year = {2011},
               pages = {3907--3914},
            keywords = {ARRAY(0x7f785945f680)},
                 url = {http://eprints.lincoln.ac.uk/6764/},
            abstract = {In this paper we present an account
    of the problems faced by a mobile robot given
    an incomplete tour of an unknown environment,
    and introduce a collection of techniques which can
    generate successful behaviour even in the presence
    of such problems. Underlying our approach is the
    principle that an autonomous system must be motivated
    to act to gather new knowledge, and to validate
    and correct existing knowledge. This principle is
    embodied in Dora, a mobile robot which features
    the aforementioned techniques: shared representations,
    non-monotonic reasoning, and goal generation
    and management. To demonstrate how well this
    collection of techniques work in real-world situations
    we present a comprehensive analysis of the Dora
    system?s performance over multiple tours in an indoor
    environment. In this analysis Dora successfully
    completed 18 of 21 attempted runs, with all but
    3 of these successes requiring one or more of the
    integrated techniques to recover from problems.}
    }
  • N. Hawes, M. Hanheide, J. Hargreaves, B. Page, H. Zender, and P. Jensfelt, “Home alone: autonomous extension and correction of spatial representations,” in Robotics and Automation (ICRA), 2011 IEEE International Conference on, Shanghai, 2011, pp. 3907-3914.
    [BibTeX] [Abstract] [EPrints]

    In this paper we present an account of the problems faced by a mobile robot given an incomplete tour of an unknown environment, and introduce a collection of techniques which can generate successful behaviour even in the presence of such problems. Underlying our approach is the principle that an autonomous system must be motivated to act to gather new knowledge, and to validate and correct existing knowledge. This principle is embodied in Dora, a mobile robot which features the aforementioned techniques: shared representations, non-monotonic reasoning, and goal generation and management. To demonstrate how well this collection of techniques work in real-world situations we present a comprehensive analysis of the Dora system’s performance over multiple tours in an indoor environment. In this analysis Dora successfully completed 18 of 21 attempted runs, with all but 3 of these successes requiring one or more of the integrated techniques to recover from problems. Â\copyright 2011 IEEE.

    @inproceedings{lirolem8353,
               month = {May},
              author = {N. Hawes and Marc Hanheide and J. Hargreaves and B. Page and H. Zender and P. Jensfelt},
                note = {Conference of 2011 IEEE International Conference on Robotics and Automation, ICRA 2011; Conference Date: 9 May 2011 through 13 May 2011; Conference Code: 94261},
           booktitle = {Robotics and Automation (ICRA), 2011 IEEE International Conference on},
               title = {Home alone: autonomous extension and correction of spatial representations},
             address = {Shanghai},
           publisher = {IEEE},
                year = {2011},
             journal = {Proceedings - IEEE International Conference on Robotics and Automation},
               pages = {3907--3914},
            keywords = {ARRAY(0x7f78594107c8)},
                 url = {http://eprints.lincoln.ac.uk/8353/},
            abstract = {In this paper we present an account of the problems faced by a mobile robot given an incomplete tour of an unknown environment, and introduce a collection of techniques which can generate successful behaviour even in the presence of such problems. Underlying our approach is the principle that an autonomous system must be motivated to act to gather new knowledge, and to validate and correct existing knowledge. This principle is embodied in Dora, a mobile robot which features the aforementioned techniques: shared representations, non-monotonic reasoning, and goal generation and management. To demonstrate how well this collection of techniques work in real-world situations we present a comprehensive analysis of the Dora system's performance over multiple tours in an indoor environment. In this analysis Dora successfully completed 18 of 21 attempted runs, with all but 3 of these successes requiring one or more of the integrated techniques to recover from problems. {\^A}{\copyright} 2011 IEEE.}
    }
  • G. Neumann, “Variational inference for policy search in changing situations,” in 28th International Conference on Machine Learning (ICML-11), 2011, pp. 817-824.
    [BibTeX] [Abstract] [EPrints]

    Many policy search algorithms minimize the Kullback-Leibler (KL) divergence to a certain target distribution in order to fit their policy. The commonly used KL-divergence forces the resulting policy to be ?reward-attracted?. The policy tries to reproduce all positively rewarded experience while negative experience is neglected. However, the KL-divergence is not symmetric and we can also minimize the the reversed KL-divergence, which is typically used in variational inference. The policy now becomes ?cost-averse?. It tries to avoid reproducing any negatively-rewarded experience while maximizing exploration. Due to this ?cost-averseness? of the policy, Variational Inference for Policy Search (VIP) has several interesting properties. It requires no kernelbandwith nor exploration rate, such settings are determined automatically by the inference. The algorithm meets the performance of state-of-theart methods while being applicable to simultaneously learning in multiple situations. We concentrate on using VIP for policy search in robotics. We apply our algorithm to learn dynamic counterbalancing of different kinds of pushes with human-like 2-link and 4-link robots.

    @inproceedings{lirolem25793,
           booktitle = {28th International Conference on Machine Learning (ICML-11)},
               month = {June},
               title = {Variational inference for policy search in changing situations},
              author = {Gerhard Neumann},
                year = {2011},
               pages = {817--824},
             journal = {Proceedings of the 28th International Conference on Machine Learning, ICML 2011},
            keywords = {ARRAY(0x7f7859462560)},
                 url = {http://eprints.lincoln.ac.uk/25793/},
            abstract = {Many policy search algorithms minimize the Kullback-Leibler (KL) divergence to a certain
    target distribution in order to fit their policy. The commonly used KL-divergence forces the resulting
    policy to be ?reward-attracted?. The policy tries to reproduce all positively rewarded experience
    while negative experience is neglected. However, the KL-divergence is not symmetric
    and we can also minimize the the reversed KL-divergence, which is typically used in variational
    inference. The policy now becomes ?cost-averse?. It tries to avoid reproducing any negatively-rewarded experience while maximizing exploration. Due to this ?cost-averseness? of the policy, Variational Inference for Policy Search (VIP) has several interesting properties. It requires no kernelbandwith nor exploration rate, such settings are
    determined automatically by the inference. The algorithm meets the performance of state-of-theart
    methods while being applicable to simultaneously learning in multiple situations. We concentrate on using VIP for policy search in robotics. We apply our algorithm to learn dynamic counterbalancing of different kinds of
    pushes with human-like 2-link and 4-link robots.}
    }
  • A. Peters, T. P. Spexard, M. Hanheide, and P. Weiss, “Hey robot, get out of my way: survey on a spatial and situational movement concept in HRI,” in Behaviour Monitoring and Interpretation – BMI Well-being, B. Gottfried and H. Aghajan, Eds., IOS Press, 2011.
    [BibTeX] [Abstract] [EPrints]

    Mobile robots are already applied in factories and hospitals, merely to do a distinct task. It is envisioned that robots assist in households soon. Those service robots will have to cope with several situations and tasks and of course with sophisticated human-robot interactions (HRI). Therefore, a robot has not only to consider social rules with respect to proxemics, it must detect in which (interaction) situation it is in and act accordingly. With respect to spatial HRI, we concentrate on the use of non-verbal communication. This chapter stresses the meaning of both, machine movements as signals towards a human and human body language. Considering these aspects will make interaction simpler and smoother. An observational study is presented to acquire a concept of spatial prompting by a robot and by a human. When a person and robot meet in a narrow hallway in order to pass by, they have to make room for each other. But how can a robot make sure that both really want to pass by instead of starting interaction? This especially concerns narrow, non-artificial surroundings. Which social signals are expected by the user and on the other side, can be generated or processed by a robot? The results will show what an appropriate passing behaviour is and how to distinguish between passage situations and others. The results shed light upon the readability of signals in spatial HRI.

    @incollection{lirolem6714,
           booktitle = {Behaviour Monitoring and Interpretation - BMI Well-being},
              editor = {B. Gottfried and H. Aghajan},
               month = {April},
               title = {Hey robot, get out of my way: survey on a spatial and situational movement concept in HRI},
              author = {Annika Peters and Thorsten P. Spexard and Marc Hanheide and Petra Weiss},
           publisher = {IOS Press},
                year = {2011},
            keywords = {ARRAY(0x7f7859435a28)},
                 url = {http://eprints.lincoln.ac.uk/6714/},
            abstract = {Mobile robots are already applied in factories and hospitals, merely to do a distinct task. It is envisioned that robots assist in households soon. Those service robots will have to cope with several situations and tasks and of course with sophisticated human-robot interactions (HRI). Therefore, a robot has not only to consider social rules with respect to proxemics, it must detect in which (interaction) situation it is in and act accordingly. With respect to spatial HRI, we concentrate on the use of non-verbal communication. This chapter stresses the meaning of both, machine movements as signals towards a human and human body language. Considering these aspects will make interaction simpler and smoother. An observational study is presented to acquire a concept of spatial prompting by a robot and by a human. When a person and robot meet in a narrow hallway in order to pass by, they have to make room for each other. But how can a robot make sure that both really want to pass by instead of starting interaction? This especially concerns narrow, non-artificial surroundings. Which social signals are expected by the user and on the other side, can be generated or processed by a robot? The results will show what an appropriate passing behaviour is and how to distinguish between passage situations and others. The results shed light upon the readability of signals in spatial HRI.}
    }
  • M. Smith, M. Shaker, S. Yue, and T. Duckett, “AltURI: a thin middleware for simulated robot vision applications,” in IEEE International Conference on Computer Science and information Technology (ICCSIT), 2011.
    [BibTeX] [Abstract] [EPrints]

    Fast software performance is often the focus when developing real-time vision-based control applications for robot simulators. In this paper we have developed a thin, high performance middleware for USARSim and other simulators designed for real-time vision-based control applications. It includes a fast image server providing images in OpenCV, Matlab or web formats and a simple command/sensor processor. The interface has been tested in USARSim with an Unmanned Aerial Vehicle using two control applications; landing using a reinforcement learning algorithm and altitude control using elementary motion detection. The middleware has been found to be fast enough to control the flying robot as well as very easy to set up and use.

    @inproceedings{lirolem4824,
           booktitle = {IEEE International Conference on Computer Science and information Technology (ICCSIT)},
               month = {June},
               title = {AltURI: a thin middleware for simulated robot vision applications},
              author = {Mark Smith and Marwan Shaker and Shigang Yue and Tom Duckett},
           publisher = {IEEE},
                year = {2011},
            keywords = {ARRAY(0x7f78593b5bc0)},
                 url = {http://eprints.lincoln.ac.uk/4824/},
            abstract = {Fast software performance is often the focus when developing real-time vision-based control applications for robot simulators. In this paper we have developed a thin, high performance middleware for USARSim and other simulators designed for real-time vision-based control applications. It includes a fast image server providing images in OpenCV, Matlab or web formats and a simple command/sensor processor. The interface has been tested in USARSim with an Unmanned Aerial Vehicle using two control applications; landing using a reinforcement learning algorithm and altitude control using elementary motion detection. The middleware has been found to be fast enough to control the flying robot as well as very easy to set up and use.}
    }
  • O. Szymanezyk, P. Dickinson, and T. Duckett, “Towards agent-based crowd simulation in airports using games technology,” in Agent and multi-agent systems: technologies and applications, Berling Heidelberg: Springer-Verlag, 2011, vol. 6682, pp. 524-533.
    [BibTeX] [Abstract] [EPrints]

    We adapt popular video games technology for an agent-based crowd simulation in an airport terminal. To achieve this, we investigate the unique traits of airports and implement a virtual crowd by exploiting a scalable layered intelligence technique in combination with physics middleware and a socialforces approach. Our experiments show that the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating navigation behaviour.

    @incollection{lirolem4569,
              volume = {6682},
              number = {6682},
               month = {September},
              author = {Oliver Szymanezyk and Patrick Dickinson and Tom Duckett},
              series = {Lecture Notes in Computer Science},
           booktitle = {Agent and multi-agent systems: technologies and applications},
               title = {Towards agent-based crowd simulation in airports using games technology},
             address = {Berling Heidelberg},
           publisher = {Springer-Verlag},
                year = {2011},
               pages = {524--533},
            keywords = {ARRAY(0x7f78592e4a68)},
                 url = {http://eprints.lincoln.ac.uk/4569/},
            abstract = {We adapt popular video games technology for an agent-based crowd simulation in an airport terminal. To achieve this, we investigate the unique traits of airports and implement a virtual crowd by exploiting a scalable layered intelligence technique in combination with physics middleware and a socialforces approach. Our experiments show that the framework runs at interactive frame-rate and evaluate the scalability with increasing number of agents demonstrating
    navigation behaviour.}
    }
  • O. Szymanezyk, P. Dickinson, and T. Duckett, “From individual characters to large crowds: augmenting the believability of open-world games through exploring social emotion in pedestrian groups,” in Think Design Play: DiGRA Conference, 2011.
    [BibTeX] [Abstract] [EPrints]

    Crowds of non-player characters improve the game-play experiences of open-world video-games. Grouping is a common phenomenon of crowds and plays an important role in crowd behaviour. Recent crowd simulation research focuses on group modelling in pedestrian crowds and game-designers have argued that the design of non-player characters should capture and exploit the relationship between characters. The concepts of social groups and inter-character relationships are not new in social psychology, and on-going work addresses the social life of emotions and its behavioural consequences on individuals and groups alike. The aim of this paper is to provide an overview of current research in social psychology, and to use the findings as a source of inspiration to design a social network of non-player characters, with application to the problem of group modelling in simulated crowds in computer games.

    @inproceedings{lirolem4662,
           booktitle = {Think Design Play: DiGRA Conference},
               month = {September},
               title = {From individual characters to large crowds: augmenting the believability of open-world games through exploring social emotion in pedestrian groups},
              author = {Oliver Szymanezyk and Patrick Dickinson and Tom Duckett},
           publisher = {ARA Digital Media Private Limited},
                year = {2011},
            keywords = {ARRAY(0x7f7858f6f0b8)},
                 url = {http://eprints.lincoln.ac.uk/4662/},
            abstract = {Crowds of non-player characters improve the game-play experiences of open-world video-games. Grouping is a common phenomenon of crowds and plays an important role in crowd behaviour. Recent crowd simulation research focuses on group modelling in pedestrian crowds and game-designers have argued that the design of non-player characters should capture and exploit the relationship between characters. The concepts of social groups and inter-character relationships are not new in social psychology, and on-going work addresses the social life of emotions and its behavioural consequences on individuals and groups alike. The aim of this paper is to provide an overview of current research in social psychology, and to use the findings as a source of inspiration to design a social network of non-player characters, with application to the problem of group modelling in simulated crowds in computer games.}
    }
  • M. L. Walters, M. Lohse, M. Hanheide, B. Wrede, D. S. Syrdal, K. L. Koay, A. Green, H. Huttenrauch, K. Dautenhahn, G. Sagerer, and K. Severinson-Eklundh, “Evaluating the robot personality and verbal behavior of domestic robots using video-based studies,” Advanced Robotics, vol. 25, iss. 18, pp. 2233-2254, 2011.
    [BibTeX] [Abstract] [EPrints]

    Robots are increasingly being used in domestic environments and should be able to interact with inexperienced users. Human-human interaction and human-computer interaction research findings are relevant, but often limited because robots are different from both humans and computers. Therefore, new human-robot interaction (HRI) research methods can inform the design of robots suitable for inexperienced users. A video-based HRI (VHRI) methodology was here used to carry out a multi-national HRI user study for the prototype domestic robot BIRON (BIelefeld RObot companioN). Previously, the VHRI methodology was used in constrained HRI situations, while in this study HRIs involved a series of events as part of a ‘hometour’ scenario. Thus, the present work is the first study of this methodology in extended HRI contexts with a multi-national approach. Participants watched videos of the robot interacting with a human actor and rated two robot behaviors (Extrovert and Introvert). Participants’ perceptions and ratings of the robot’s behaviors differed with regard to both verbal interactions and person following by the robot. The study also confirms that the VHRI methodology provides a valuable means to obtain early user feedback, even before fully working prototypes are available. This can usefully guide the future design work on robots, and associated verbal and non-verbal behaviors.

    @article{lirolem6560,
              volume = {25},
              number = {18},
               month = {December},
              author = {Michael L. Walters and Manja Lohse and Marc Hanheide and Britte Wrede and Dag Sverre Syrdal and Kheng Lee Koay and Anders Green and Helge Huttenrauch and Kerstin Dautenhahn and Gerhard Sagerer and Kerstin Severinson-Eklundh},
                note = {Robots are increasingly being used in domestic environments and should be able to interact with inexperienced users. Human-human interaction and human-computer interaction research findings are relevant, but often limited because robots are different from both humans and computers. Therefore, new human-robot interaction (HRI) research methods can inform the design of robots suitable for inexperienced users. A video-based HRI (VHRI) methodology was here used to carry out a multi-national HRI user study for the prototype domestic robot BIRON (BIelefeld RObot companioN). Previously, the VHRI methodology was used in constrained HRI situations, while in this study HRIs involved a series of events as part of a 'hometour' scenario. Thus, the present work is the first study of this methodology in extended HRI contexts with a multi-national approach. Participants watched videos of the robot interacting with a human actor and rated two robot behaviors (Extrovert and Introvert). Participants' perceptions and ratings of the robot's behaviors differed with regard to both verbal interactions and person following by the robot. The study also confirms that the VHRI methodology provides a valuable means to obtain early user feedback, even before fully working prototypes are available. This can usefully guide the future design work on robots, and associated verbal and non-verbal behaviors.},
               title = {Evaluating the robot personality and verbal behavior of domestic robots using video-based studies},
           publisher = {Taylor \& Francis},
                year = {2011},
             journal = {Advanced Robotics},
               pages = {2233--2254},
            keywords = {ARRAY(0x7f78594099a0)},
                 url = {http://eprints.lincoln.ac.uk/6560/},
            abstract = {Robots are increasingly being used in domestic environments and should be able to interact with inexperienced users. Human-human interaction and human-computer interaction research findings are relevant, but often limited because robots are different from both humans and computers. Therefore, new human-robot interaction (HRI) research methods can inform the design of robots suitable for inexperienced users. A video-based HRI (VHRI) methodology was here used to carry out a multi-national HRI user study for the prototype domestic robot BIRON (BIelefeld RObot companioN). Previously, the VHRI methodology was used in constrained HRI situations, while in this study HRIs involved a series of events as part of a 'hometour' scenario. Thus, the present work is the first study of this methodology in extended HRI contexts with a multi-national approach. Participants watched videos of the robot interacting with a human actor and rated two robot behaviors (Extrovert and Introvert). Participants' perceptions and ratings of the robot's behaviors differed with regard to both verbal interactions and person following by the robot. The study also confirms that the VHRI methodology provides a valuable means to obtain early user feedback, even before fully working prototypes are available. This can usefully guide the future design work on robots, and associated verbal and non-verbal behaviors.}
    }
  • S. Yue, H. Wei, M. Li, Q. Liang, and L. Wang, “ICNC-FSKD 2010 special issue on computers & mathematics in natural computation and knowledge discovery,” Computers and Mathematics with Applications, vol. 62, iss. 7, pp. 2683-2684, 2011.
    [BibTeX] [Abstract] [EPrints]

    Natural computation, as an exciting and emerging interdisciplinary field, has been witnessing a surge of newly developed theories, methodologies and applications in recent years. These innovations have generated a huge impact in tackling complex and challenging real world problems. Not only are the well established intelligent techniques, such as neural networks, fuzzy systems, genetic and evolutionary algorithms, and cellular automata expanding to new application areas; the new forms of natural computation that have emerged recently, for example, swarm intelligence, artificial immune systems, bio-molecular computing and membrane computing, quantum computing, and granular computing, are also providing additional tools for various applications. One attractive area that natural computation has been playing a major role in is knowledge discovery. There are many success stories on natural computation and knowledge discovery, as you will find out in this special issue

    @article{lirolem10329,
              volume = {62},
              number = {7},
               month = {October},
              author = {Shigang Yue and Hua-Liang Wei and Maozhen Li and Qilian Liang and Lipo Wang},
               title = {ICNC-FSKD 2010 special issue on computers \& mathematics in natural computation and knowledge discovery},
           publisher = {Elsevier},
                year = {2011},
             journal = {Computers and Mathematics with Applications},
               pages = {2683--2684},
            keywords = {ARRAY(0x7f78593a0578)},
                 url = {http://eprints.lincoln.ac.uk/10329/},
            abstract = {Natural computation, as an exciting and emerging interdisciplinary field, has been witnessing a surge of newly developed theories, methodologies and applications in recent years. These innovations have generated a huge impact in tackling complex and challenging real world problems. Not only are the well established intelligent techniques, such as neural networks, fuzzy systems, genetic and evolutionary algorithms, and cellular automata expanding to new application areas; the new forms of natural computation that have emerged recently, for example, swarm intelligence, artificial immune systems, bio-molecular computing and membrane computing, quantum computing, and granular computing, are also providing additional tools for various applications. One attractive area that natural computation has been playing a major role in is knowledge discovery. There are many success stories on natural computation and knowledge discovery, as you will find out in this special issue}
    }

2010

  • F. Arvin, K. Samsudin, and R. Ramli, “Development of IR-based short-range communication techniques for swarm robot applications,” Advances in Electrical and Computer Engineering, vol. 10, iss. 4, pp. 61-68, 2010.
    [BibTeX] [Abstract] [EPrints]

    This paper proposes several designs for a reliable infra-red based communication techniques for swarm robotic applications. The communication system was deployed on an autonomous miniature mobile robot (AMiR), a swarm robotic platform developed earlier. In swarm applications, all participating robots must be able to communicate and share data. Hence a suitable communication medium and a reliable technique are required. This work uses infrared radiation for transmission of swarm robots messages. Infrared transmission methods such as amplitude and frequency modulations will be presented along with experimental results. Finally the effects of the modulation techniques and other parameters on collective behavior of swarm robots will be analyzed.

    @article{lirolem5796,
              volume = {10},
              number = {4},
               month = {December},
              author = {Farshad Arvin and Khairulmizam Samsudin and Rahman Ramli},
               title = {Development of IR-based short-range communication techniques for swarm robot applications},
           publisher = {AECE},
                year = {2010},
             journal = {Advances in Electrical and Computer Engineering},
               pages = {61--68},
            keywords = {ARRAY(0x7f7859464920)},
                 url = {http://eprints.lincoln.ac.uk/5796/},
            abstract = {This paper proposes several designs for a reliable infra-red based communication techniques for swarm robotic applications. The communication system was deployed on an autonomous miniature mobile robot (AMiR), a swarm robotic platform developed earlier. In swarm applications, all participating robots must be able to communicate and share data. Hence a suitable communication medium and a reliable technique are required. This work uses infrared radiation for transmission of swarm robots messages. Infrared transmission methods such as amplitude and frequency modulations will be presented along with experimental results. Finally the effects of the modulation techniques and other parameters on collective behavior of swarm robots will be analyzed.}
    }
  • F. Arvin, S. Doraisamy, K. Samsudin, and A. R. Ramli, “Self-localization of swarm robots based on voice signal acquisition,” in International Conference on Computer and Communication Engineering (ICCCE), 2010, pp. 1-5.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an acoustical signal tracking experiment by swarm mobile robots. Biological swarm is a fascinating behavior of nature which was inspired from social insects’ behavior. A mobile robot that is designed as a swarm robotic platform was employed for implementing voice exploration behavior. An additional module was developed to connect to robots for processing given voice signals using the proportional signal strength approach that estimates orientation of sound source using fuzzy logic approach. The voice processor module utilizes four condenser microphones with around -47db sensitivity which are placed in different directions of the circuit board for capturing surrounding sound signals. Captured samples by microphones are processed to estimate the relative positions of the sound source in the robotic environment. After estimating the position of the signal’s source, all participants move towards similar to the insects’ colony. The participant robots have an individual task for the estimation of source location from captured samples. Moreover, according to the swarm definition, an additional cooperation between swarm participants is required to achieve a correct colony of robots. Obtained results illustrate the feasibility of the proposed technique and hardware interface for sound signals acquisition with swarm robots.

    @inproceedings{lirolem11356,
           booktitle = {International Conference on Computer and Communication Engineering (ICCCE)},
               month = {May},
               title = {Self-localization of swarm robots based on voice signal acquisition},
              author = {Farshad Arvin and Shyamala Doraisamy and Khairulmizam Samsudin and Abdul Rahman Ramli},
           publisher = {IEEE},
                year = {2010},
               pages = {1--5},
            keywords = {ARRAY(0x7f7858f7d4e0)},
                 url = {http://eprints.lincoln.ac.uk/11356/},
            abstract = {This paper presents an acoustical signal tracking experiment by swarm mobile robots. Biological swarm is a fascinating behavior of nature which was inspired from social insects' behavior. A mobile robot that is designed as a swarm robotic platform was employed for implementing voice exploration behavior. An additional module was developed to connect to robots for processing given voice signals using the proportional signal strength approach that estimates orientation of sound source using fuzzy logic approach. The voice processor module utilizes four condenser microphones with around -47db sensitivity which are placed in different directions of the circuit board for capturing surrounding sound signals. Captured samples by microphones are processed to estimate the relative positions of the sound source in the robotic environment. After estimating the position of the signal's source, all participants move towards similar to the insects' colony. The participant robots have an individual task for the estimation of source location from captured samples. Moreover, according to the swarm definition, an additional cooperation between swarm participants is required to achieve a correct colony of robots. Obtained results illustrate the feasibility of the proposed technique and hardware interface for sound signals acquisition with swarm robots.}
    }
  • F. Arvin and S. Doraisamy, “Heart sound musical transcription technique using multi-Level preparation,” International Review on Computers and Software (I.Re.Co.S.), vol. 5, iss. 6, pp. 595-600, 2010.
    [BibTeX] [Abstract] [EPrints]

    Musical transcription of heart sound is a new idea to provide a textual biomedical database. Textual database allows applying several indexing and searching techniques in order to monitor patient behavior for a long duration. MIDI commands produce a semi-structural musical file format which enables to apply various applications. Main objective of this paper is the extraction of fundamental frequency of the given heart sound which is recorded with an electrical stethoscope. Based on extracted fundamental frequencies, the logarithmical relationship of pitch numbers will be estimated. Generally the captured heart sound includes several types of noises such as other organs sound and ambient voice. Hence, filtering of the heart sound is indispensable. Thus, three levels of preparation techniques which are wavelet transform, frequency limitation, and amplitude reconstruction will be applied on the heart sound sequentially. The results of the performed experiments show the accuracy of approximately 93\% $\pm$2. The statistical analyses illustrated that each level of the preparation, significantly improved the accuracy of the transcription (p \ensuremath< 0.005).

    @article{lirolem6087,
              volume = {5},
              number = {6},
               month = {November},
              author = {Farshad Arvin and Shyamala Doraisamy},
               title = {Heart sound musical transcription technique using multi-Level preparation},
           publisher = {Praise Worthy Prize},
                year = {2010},
             journal = {International Review on Computers and Software (I.Re.Co.S.)},
               pages = {595--600},
            keywords = {ARRAY(0x7f785945a170)},
                 url = {http://eprints.lincoln.ac.uk/6087/},
            abstract = {Musical transcription of heart sound is a new idea to provide a textual biomedical database. Textual database allows applying several indexing and searching techniques in order to monitor patient behavior for a long duration. MIDI commands produce a semi-structural musical file format which enables to apply various applications. Main objective of this paper is the extraction of fundamental frequency of the given heart sound which is recorded with an electrical stethoscope. Based on extracted fundamental frequencies, the logarithmical relationship of pitch numbers will be estimated. Generally the captured heart sound includes several types of noises such as other organs sound and ambient voice. Hence, filtering of the heart sound is indispensable. Thus, three levels of preparation techniques which are wavelet transform, frequency limitation, and amplitude reconstruction will be applied on the heart sound sequentially. The results of the performed experiments show the accuracy of approximately 93\% {$\pm$}2. The statistical analyses illustrated that each level of the preparation, significantly improved the accuracy of the transcription (p {\ensuremath{<}} 0.005).}
    }
  • M. Barnes, T. Duckett, G. Cielniak, G. Stroud, and G. Harper, “Visual detection of blemishes in potatoes using minimalist boosted classifiers,” Journal of Food Engineering, vol. 98, iss. 3, pp. 339-346, 2010.
    [BibTeX] [Abstract] [EPrints]

    This paper introduces novel methods for detecting blemishes in potatoes using machine vision. After segmentation of the potato from the background, a pixel-wise classifier is trained to detect blemishes using features extracted from the image. A very large set of candidate features, based on statistical information relating to the colour and texture of the region surrounding a given pixel, is first extracted. Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating between blemishes and non-blemishes. With this approach, different features can be selected for different potato varieties, while also handling the natural variation in fresh produce due to different seasons, lighting conditions, etc. The results show that the method is able to build “minimalist” classifiers that optimise detection performance at low computational cost. In experiments, blemish detectors were trained for both white and red potato varieties, achieving 89.6$\backslash$\% and 89.5$\backslash$\% accuracy, respectively.

    @article{lirolem2206,
              volume = {98},
              number = {3},
               month = {June},
              author = {Michael Barnes and Tom Duckett and Grzegorz Cielniak and Graeme Stroud and Glyn Harper},
                note = {This paper introduces novel methods for detecting blemishes in potatoes using machine vision. After segmentation of the potato from the background, a pixel-wise classifier is trained to detect blemishes using features extracted from the image.
    A very large set of candidate features, based on statistical information relating to the colour and texture of the region surrounding a given pixel, is first extracted.
    Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating between blemishes and non-blemishes.
    With this approach, different features can be selected for different potato varieties, while also handling the natural variation in fresh produce due to different seasons, lighting conditions, etc.
    The results show that the method is able to build ``minimalist'' classifiers that optimise detection performance at low computational cost.
    In experiments, blemish detectors were trained for both white and red potato varieties, achieving 89.6{$\backslash$}\% and 89.5{$\backslash$}\% accuracy, respectively.},
               title = {Visual detection of blemishes in potatoes using minimalist boosted classifiers},
           publisher = {Elsevier},
                year = {2010},
             journal = {Journal of Food Engineering},
               pages = {339--346},
            keywords = {ARRAY(0x7f78592da108)},
                 url = {http://eprints.lincoln.ac.uk/2206/},
            abstract = {This paper introduces novel methods for detecting blemishes in potatoes using machine vision. After segmentation of the potato from the background, a pixel-wise classifier is trained to detect blemishes using features extracted from the image.
    A very large set of candidate features, based on statistical information relating to the colour and texture of the region surrounding a given pixel, is first extracted.
    Then an adaptive boosting algorithm (AdaBoost) is used to automatically select the best features for discriminating between blemishes and non-blemishes.
    With this approach, different features can be selected for different potato varieties, while also handling the natural variation in fresh produce due to different seasons, lighting conditions, etc.
    The results show that the method is able to build ``minimalist'' classifiers that optimise detection performance at low computational cost.
    In experiments, blemish detectors were trained for both white and red potato varieties, achieving 89.6{$\backslash$}\% and 89.5{$\backslash$}\% accuracy, respectively.}
    }
  • M. Barnes, G. Cielniak, and T. Duckett, “Minimalist AdaBoost for blemish identification in potatoes,” in International Conference on Computer Vision and Graphics 2010, 2010, pp. 209-216.
    [BibTeX] [Abstract] [EPrints]

    We present a multi-class solution based on minimalist Ad- aBoost for identifying blemishes present in visual images of potatoes. Using training examples we use Real AdaBoost to rst reduce the fea- ture set by selecting ve features for each class, then train binary clas- siers for each class, classifying each testing example according to the binary classier with the highest certainty. Against hand-drawn ground truth data we achieve a pixel match of 83\% accuracy in white potatoes and 82\% in red potatoes. For the task of identifying which blemishes are present in each potato within typical industry dened criteria (10\% coverage) we achieve accuracy rates of 93\% and 94\%, respectively.

    @inproceedings{lirolem5517,
               month = {September},
              author = {Michael Barnes and Grzegorz Cielniak and Tom Duckett},
                note = {Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
    Volume 6374 LNCS, Issue PART 1, 2010, Pages 209-216},
           booktitle = {International Conference on Computer Vision and Graphics 2010},
               title = {Minimalist AdaBoost for blemish identification
    in potatoes},
           publisher = {Springer},
               pages = {209--216},
                year = {2010},
            keywords = {ARRAY(0x7f78593a6190)},
                 url = {http://eprints.lincoln.ac.uk/5517/},
            abstract = {We present a multi-class solution based on minimalist Ad-
    aBoost for identifying blemishes present in visual images of potatoes.
    Using training examples we use Real AdaBoost to rst reduce the fea-
    ture set by selecting ve features for each class, then train binary clas-
    siers for each class, classifying each testing example according to the
    binary classier with the highest certainty. Against hand-drawn ground
    truth data we achieve a pixel match of 83\% accuracy in white potatoes
    and 82\% in red potatoes. For the task of identifying which blemishes
    are present in each potato within typical industry dened criteria (10\%
    coverage) we achieve accuracy rates of 93\% and 94\%, respectively.}
    }
  • N. Bellotto and H. Hu, “A bank of unscented Kalman filters for multimodal human perception with mobile service robots,” International Journal of Social Robotics, vol. 2, iss. 2, pp. 121-136, 2010.
    [BibTeX] [Abstract] [EPrints]

    A new generation of mobile service robots could be ready soon to operate in human environments if they can robustly estimate position and identity of surrounding people. Researchers in this field face a number of challenging problems, among which sensor uncertainties and real-time constraints. In this paper, we propose a novel and efficient solution for simultaneous tracking and recognition of people within the observation range of a mobile robot. Multisensor techniques for legs and face detection are fused in a robust probabilistic framework to height, clothes and face recognition algorithms. The system is based on an efficient bank of Unscented Kalman Filters that keeps a multi-hypothesis estimate of the person being tracked, including the case where the latter is unknown to the robot. Several experiments with real mobile robots are presented to validate the proposed approach. They show that our solutions can improve the robot’s perception and recognition of humans, providing a useful contribution for the future application of service robotics.

    @article{lirolem2566,
              volume = {2},
              number = {2},
               month = {June},
              author = {Nicola Bellotto and Huosheng Hu},
                note = {A new generation of mobile service robots could be ready soon to operate in human environments if they can robustly estimate position and identity of surrounding people. Researchers in this field face a number of challenging problems, among which sensor uncertainties and real-time constraints.
    In this paper, we propose a novel and efficient solution for simultaneous tracking and recognition of people within the observation range of a mobile robot. Multisensor techniques for legs and face detection are fused in a robust probabilistic framework to height, clothes and face recognition algorithms. The system is based on an efficient bank of Unscented Kalman Filters that keeps a multi-hypothesis estimate of the person being tracked, including the case where the latter is unknown to the robot.
    Several experiments with real mobile robots are presented to validate the proposed approach. They show that our solutions can improve the robot's perception and recognition of humans, providing a useful contribution for the future application of service robotics.},
               title = {A bank of unscented Kalman filters for multimodal human perception with mobile service robots},
           publisher = {Springer},
                year = {2010},
             journal = {International Journal of Social Robotics},
               pages = {121--136},
            keywords = {ARRAY(0x7f78590bae58)},
                 url = {http://eprints.lincoln.ac.uk/2566/},
            abstract = {A new generation of mobile service robots could be ready soon to operate in human environments if they can robustly estimate position and identity of surrounding people. Researchers in this field face a number of challenging problems, among which sensor uncertainties and real-time constraints.
    In this paper, we propose a novel and efficient solution for simultaneous tracking and recognition of people within the observation range of a mobile robot. Multisensor techniques for legs and face detection are fused in a robust probabilistic framework to height, clothes and face recognition algorithms. The system is based on an efficient bank of Unscented Kalman Filters that keeps a multi-hypothesis estimate of the person being tracked, including the case where the latter is unknown to the robot.
    Several experiments with real mobile robots are presented to validate the proposed approach. They show that our solutions can improve the robot's perception and recognition of humans, providing a useful contribution for the future application of service robotics.}
    }
  • N. Bellotto and H. Hu, “Computationally efficient solutions for tracking people with a mobile robot: an experimental evaluation of Bayesian filters,” Autonomous Robots, vol. 28, iss. 4, pp. 425-438, 2010.
    [BibTeX] [Abstract] [EPrints]

    Modern service robots will soon become an essential part of modern society. As they have to move and act in human environments, it is essential for them to be provided with a fast and reliable tracking system that localizes people in the neighbourhood. It is therefore important to select the most appropriate filter to estimate the position of these persons. This paper presents three efficient implementations of multisensor-human tracking based on different Bayesian estimators: Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF) and Sampling Importance Resampling (SIR) particle filter. The system implemented on a mobile robot is explained, introducing the methods used to detect and estimate the position of multiple people. Then, the solutions based on the three filters are discussed in detail. Several real experiments are conducted to evaluate their performance, which is compared in terms of accuracy, robustness and execution time of the estimation. The results show that a solution based on the UKF can perform as good as particle filters and can be often a better choice when computational efficiency is a key issue.

    @article{lirolem2286,
              volume = {28},
              number = {4},
               month = {May},
              author = {Nicola Bellotto and Huosheng Hu},
               title = {Computationally efficient solutions for tracking people with a mobile robot: an experimental evaluation of Bayesian filters},
           publisher = {Springer},
                year = {2010},
             journal = {Autonomous Robots},
               pages = {425--438},
            keywords = {ARRAY(0x7f7858f540f8)},
                 url = {http://eprints.lincoln.ac.uk/2286/},
            abstract = {Modern service robots will soon become an essential part of modern society. As they have to move and act in human environments, it is essential for them to be provided with a fast and reliable tracking system that localizes people in the neighbourhood. It is therefore important to select the most appropriate filter to estimate the position of these persons.
    This paper presents three efficient implementations of multisensor-human tracking based on different Bayesian estimators: Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF) and Sampling Importance Resampling (SIR) particle filter. The system implemented on a mobile robot is explained, introducing the methods used to detect and estimate the position of multiple people. Then, the solutions based on the three filters are discussed in detail. Several real experiments are conducted to evaluate their performance, which is compared in terms of accuracy, robustness and execution time of the estimation. The results show that a solution based on the UKF can perform as good as particle filters and can be often a better choice when computational efficiency is a key issue.}
    }
  • G. Cielniak, T. Duckett, and A. J. Lilienthal, “Data association and occlusion handling for vision-based people tracking by mobile robots,” Robotics and Autonomous Systems, vol. 58, iss. 5, pp. 435-443, 2010.
    [BibTeX] [Abstract] [EPrints]

    This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets.

    @article{lirolem2277,
              volume = {58},
              number = {5},
               month = {May},
              author = {Grzegorz Cielniak and Tom Duckett and Achim J. Lilienthal},
               title = {Data association and occlusion handling for vision-based people tracking by mobile robots},
           publisher = {Elsevier B.V.},
                year = {2010},
             journal = {Robotics and Autonomous Systems},
               pages = {435--443},
            keywords = {ARRAY(0x7f7859449018)},
                 url = {http://eprints.lincoln.ac.uk/2277/},
            abstract = {This paper presents an approach for tracking multiple persons on a mobile robot with a combination of colour and thermal vision sensors, using several new techniques. First, an adaptive colour model is incorporated into the measurement model of the tracker. Second, a new approach for detecting occlusions is introduced, using a machine learning classifier for pairwise comparison of persons (classifying which one is in front of the other). Third, explicit occlusion handling is incorporated into the tracker. The paper presents a comprehensive, quantitative evaluation of the whole system and its different components using several real world data sets.}
    }
  • H. Cuayahuitl, S. Renals, O. Lemon, and H. Shimodaira, “Evaluation of a hierarchical reinforcement learning spoken dialogue system,” Computer Speech & Language, vol. 24, iss. 2, pp. 395-429, 2010.
    [BibTeX] [Abstract] [EPrints]

    We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ?Precision-Recall?. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20\%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems.

    @article{lirolem22208,
              volume = {24},
              number = {2},
               month = {April},
              author = {Heriberto Cuayahuitl and Steve Renals and Oliver Lemon and Hiroshi Shimodaira},
               title = {Evaluation of a hierarchical reinforcement learning spoken dialogue system},
           publisher = {Elsevier for International Speech Communication Association (ISCA)},
                year = {2010},
             journal = {Computer Speech \& Language},
               pages = {395--429},
            keywords = {ARRAY(0x7f785657c6d8)},
                 url = {http://eprints.lincoln.ac.uk/22208/},
            abstract = {We describe an evaluation of spoken dialogue strategies designed using hierarchical reinforcement learning agents. The dialogue strategies were learnt in a simulated environment and tested in a laboratory setting with 32 users. These dialogues were used to evaluate three types of machine dialogue behaviour: hand-coded, fully-learnt and semi-learnt. These experiments also served to evaluate the realism of simulated dialogues using two proposed metrics contrasted with ?Precision-Recall?. The learnt dialogue behaviours used the Semi-Markov Decision Process (SMDP) model, and we report the first evaluation of this model in a realistic conversational environment. Experimental results in the travel planning domain provide evidence to support the following claims: (a) hierarchical semi-learnt dialogue agents are a better alternative (with higher overall performance) than deterministic or fully-learnt behaviour; (b) spoken dialogue strategies learnt with highly coherent user behaviour and conservative recognition error rates (keyword error rate of 20\%) can outperform a reasonable hand-coded strategy; and (c) hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of optimized dialogue behaviours in larger-scale systems.}
    }
  • F. Dayoub, T. Duckett, and G. Cielniak, “Toward an object-based semantic memory for long-term operation of mobile service robots,” in Workshop on Semantic Mapping and Autonomous Knowledge Acquisition, 2010.
    [BibTeX] [Abstract] [EPrints]

    Throughout a lifetime of operation, a mobile service robot needs to acquire, store and update its knowledge of a working environment. This includes the ability to identify and track objects in different places, as well as using this information for interaction with humans. This paper introduces a long-term updating mechanism, inspired by the modal model of human memory, to enable a mobile robot to maintain its knowledge of a changing environment. The memory model is integrated with a hybrid map that represents the global topology and local geometry of the environment, as well as the respective 3D location of objects. We aim to enable the robot to use this knowledge to help humans by suggesting the most likely locations of specific objects in its map. An experiment using omni-directional vision demonstrates the ability to track the movements of several objects in a dynamic environment over an extended period of time.

    @inproceedings{lirolem3866,
           booktitle = {Workshop on Semantic Mapping and Autonomous Knowledge Acquisition},
               month = {October},
               title = {Toward an object-based semantic memory for long-term operation of mobile service robots},
              author = {Feras Dayoub and Tom Duckett and Grzegorz Cielniak},
                year = {2010},
                note = {Throughout a lifetime of operation, a mobile service robot needs to acquire, store and update its knowledge of a working environment. This includes the ability to identify and track objects in different places, as well as using this information for interaction with humans. This paper introduces a long-term updating mechanism, inspired by the modal model of human memory, to enable a mobile robot to maintain its knowledge of a changing environment. The memory model is integrated with a hybrid map that represents the global topology and local geometry of the environment, as well as the respective 3D location of objects. We aim to enable the robot to use this knowledge to help humans by suggesting the most likely locations of specific objects in its map. An experiment using omni-directional vision demonstrates the ability to track the movements of several objects in a dynamic environment over an extended period of time.},
            keywords = {ARRAY(0x7f78592c6710)},
                 url = {http://eprints.lincoln.ac.uk/3866/},
            abstract = {Throughout a lifetime of operation, a mobile service robot needs to acquire, store and update its knowledge of a working environment. This includes the ability to identify and track objects in different places, as well as using this information for interaction with humans. This paper introduces a long-term updating mechanism, inspired by the modal model of human memory, to enable a mobile robot to maintain its knowledge of a changing environment. The memory model is integrated with a hybrid map that represents the global topology and local geometry of the environment, as well as the respective 3D location of objects. We aim to enable the robot to use this knowledge to help humans by suggesting the most likely locations of specific objects in its map. An experiment using omni-directional vision demonstrates the ability to track the movements of several objects in a dynamic environment over an extended period of time.}
    }
  • F. Dayoub, T. Duckett, and G. Cielniak, “Short- and long-term adaptation of visual place memories for mobile robots,” in International Symposium on Remembering Who We Are – Human Memory for Artificial Agents – A Symposium at the AISB 2010 Convention, Leicester, 2010, pp. 21-26.
    [BibTeX] [Abstract] [EPrints]

    This paper presents a robotic implementation of a human-inspired memory model for long-t