Available under a Creative Commons Attribution Non-Commercial Share Alike 4.0 International Licence
Computer Sciences, Information Science
Activity discovery is a challenging machine learning problem where we seek to uncover new or altered behavioural patterns in sensor data. In this paper we motivate and introduce a novel approach to evaluating activity discovery systems. Pre-annotated ground truths, often used to evaluate the performance of such systems on existing datasets, may exist at different levels of abstraction to the output of the output produced by the system. We propose a method for detecting and dealing with this situation, allowing for useful ground truth comparisons. This work has applications for activity discovery, and also for related fields. For example, it could be used to evaluate systems intended for anomaly detection, intrusion detection, automated music transcription and potentially other applications.
Rogers, E., Ross, R.J. & Kelleher, J.D. (2018). Evaluating sequence discovery systems in an abstraction-aware manner. Published in Artificial Intelligence Applications and Innovations, 14th IFIP WG 12.5 International Conference, AIAI 2018, Rhodes, Greece, May 25–27, Proceedings, pp. 261--272. doi:10.1007/978-3-319-92007-8_23