Some of the datasets provide an easy way of use through our automated benchmarking system for robotics experiments. The system is based on open source, freely-available tools commonly used in software development. While it allows for a seamless and fair comparison of a newly developed method with the original one, it does not require disclosure of neither the original codes nor evaluation datasets. Apart from the description of the system, we provide two use cases, where the researchers were able to compare their methods to the original ones in a matter of minutes without running the method on their own hardware.
|
| Automated benchmarking system workflow. |
Read more in our paper. We attached a bibtex record for your convenience.