Real-Time Analytics for Smart City Video Surveillance



Matthew Collison


Teaching Fellow - School of Computing, Newcastle University

Abstract: CCTV and IP cameras have been deployed on a mass scale over recent decades. Computer vision and big data initiatives have also provided multiple technically advanced solutions for video analysis and distributed computing. However, surveillance automation remains a challenging problem where integration and configuration of machine learning models with existing video surveillance hardware poses significant problems in data volume, bounding model complexity and latency. Here, we present a real-time video analysis pipeline for smart video surveillance that can be retro fitted to existing video streams. Our end-to-end analytics pipeline uses open source video analysis software that are performant, reliable and scalable. We leverage openCV, Kafka, Flink, Spark, Neo4J and Terraform to provide object detection and tracking, scalability, activity detection, event classification, data representation and system deployment respectively. Our design optimises compute workload through edge computing and annotation based unsupervised event classification in a microservice cloud architecture. This talk will also document a use case to demonstrate the deployment cycle for our pipeline using footage from the popular Abby Road crossing site. Our use case demonstrates how unsupervised clustering provides a generic foundation to map a user defined rule based system with event classification which is adaptable to the deployment scenario and severity level of event notifications. This use case is presented with performance benchmarking to show the architecture is appropriate for applications including small to medium size domestic surveillance and has the scope to scale to the capacity of smart cities.

Bio: I am a teaching fellow in Computer Science at Newcastle University. I have taught a range of research led specialisations within Computing and recently moved towards an emphasis on Information Systems and Data Science. My research focusses on engineering systems for data intensive challenges and includes applications of graph databases and analytics pipelines for drug discovery in gut bacteria, using real-time video streams as sensor data for smart city applications and client-side mining as a new monetisation mechanism for consumption of digital media.