Anomaly Detection in Video Surveillance: A Novel Approach Based on Sub-Trajectory Duc Vinh Ngo1 and Nang Toan Do2 and Luong Anh Tuan Nguyen3 Hanoi University of Industry, Vietnam Email: ngoducvinh@haui.edu.vn The Information Technology Institute – Vietnam National University Email: dntoan@vnu.edu.vn Ho Chi Minh City University of Transport, Vietnam Email: nlatuan@hcmutrans.edu.vn Abstract— In video surveillance based on motion trajectory, a moving object is considered abnormal if the distance from it to trajectory is greater than a threshold If a trajectory is abnormal with sub-trajectory, it will be abnormal with entire trajectory In this paper, a novel approach proposes to detect abnormal based on sub-trajectory with modified Hausdorff distance Moreover, the anomaly detection based on sub-trajectory that can be done with complete trajectory and incomplete trajectory The proposed technique is evaluated with 1000 datasets and each dataset consists of 260 trajectories The result show that the technique detect abnormal with the time faster abnormal in sub-trajectories The algorithm can detect abnormal with incomplete trajectory with the aim to reduce the detection time, so the technique can meet the video surveillance system in real-time The rest of this paper is organized as follows: Section II presents system design Section III evaluates the result of the method Finally, Section IV concludes the paper and figures out the future works Index Terms—anomaly detection, sub-trjactory, Hausdorff distance, motion trajectory I INTRODUCTION Continuous monitoring mission and ensure credibility with a large number of video streams is a challenge for the operating of monitoring system Automatic video surveillance can help reduce the cost of labor, as well as giving the appropriate notice as necessary Because of these, anomaly detection in video surveillance has attracted many researchers in the field of computer vision There are many methods of anomaly detection, but it can be classified into two groups, based on the characteristics of the line video images [1] [2] and based on analysis of the motion trajectory of object [3] [4] In recent years, the method of analysis based on the motion trajectory has received a lot of attention of the researchers [5] [6] [7] The anomaly detection technique based on trajectory analysis was done by clustering the trajectory to eliminate outliers [8] Then the abnormal is detected by calculating the distance of the new trajectory to the center of clusters The processing model is shown in Figure Most of the proposed algorithm detect abnormal with complete trajectory This is clearly a disadvantage in automatic monitoring applications with real-time In this paper, we propose a novel technique with modified Hausdorff distance [9, 10] to satisfy the properties of a metric , and segment a trajectory into subtrajectories based on the changing of the velocity [11, 12, 13, 14] Anomaly detection algorithm in video surveillance based on motion trajectory is proposed in the paper by modifying Hausdorff distance to detect Figure The processing model for anomaly detection II SYSTEM DESIGN A Some Definitions 1) Definiton 1: Distance from a point to a set Let (X, d) be complete metric space and let H(x) be compact subset of X With x ϵ X and B ϵ H(X), The distance from a point to a set is defined as follows: d(x, B) = min{d(x, y) : y ϵ B} 2) Definiion 2: Distance between two sets Let (X, d) be complete metric space With A, B ϵ H(X), The distance from set A to set B is defined as follows: d(A, B) = max{d(x, B), x ϵ A} 3) Definiion 3: Hausdorff Distance Let (X, d) be complete metric space The Haudorff distance from set A to set B is defined as follows: h(A, B) = max{d(A, B), d(B, A)} 4) Theorem h is metric on H(x) Proof If h satisfies reflexivity, symmetry and triangle inequality, then h is metric on H(x) (i) Reflexivity h(A, A) = max{d(A, A), d(A, A)} = max{d(a, A): aϵA}= (ii) Symmetry h(A, B)=max{ d(A, B), d(B, A)} = max{ d(B, A), d(A, B)} = h(B, A) (iii) Triangle inequality A≠B≠C ϵ H(x) => any aϵA, aB : d(a, B)>0 => h(A, B)≥d(a, B)>0 a ϵ A and c ϵ C, we have d(a, B) = min{d(a, b) : bϵB } ≤ min{d(a, c) + d(c, b): b ϵ B} Ö d(a, B) ≤ d(a, C) + min{d(c, b) : b ϵ B}, c ϵ C Ö d(a, B) ≤ d(a, C) + max{min{d(c, b) : bϵB}: cϵC} Ö d(a, B) ≤ d(a, C) + d(C, B) Thus, d(A, B) = max{d(a, B): aϵA} ≤ d(a, C)+d(C, B) ≤ d(A, C) + d(C, B) Similarly, we have d(B, A) ≤ d(B, C) + d(C, A) h(A, B) = max{d(A, B), d(B, A)} ≤ max{d(A, C) + d (C, B), d(B, C) + d(C, A)} ≤ max{d(A, C), d(C, A)}+max{d(C, B),d(B, C)} ≤ h(A, C) + h (C, B) 5) Definiton 5: Motion Trajectory Let O = {t1, t2, …, tn} be the motion trajectory of object O Sequence presents the position of object O at the time t1, t2, , tn Figure shows that the motion trajectory of the object was obtained in the process of tracking objects d0 (ai , b j ) vai vbj (8) vai vb j Wherein, the velocity vai and vb j are calculated by (9) and (10) vai ( xia xia 1, yia yia ) (9) vbj ( xbj xbj 1, ybj ybj ) (10) 7) Definiton 6: Route Given a collection of trajectories Rt={O1, O2, …, Or} and threshold V Rt is called a route if h(Oi, Oj) ≤ V, Oi, Oj ϵ Rt The tracking of motion established the route is shown in Figure Figure The tracking of motion objects Figure - The motion trajectory of the object 6) Definiton 6: Hausdorff Distance Between Two Trajectories Given two trajectories A={a1, a2, …, an} and B={b1, b2, …, bm} Distance between two trajectories h(A, B) is defined by (1) h(A, B) = max{d(A, B), d(B, A)} (1) Wherein, d(A, B) and d(B, A) are calculated by (2) and (3) d(A, B) = max{d(ai, B), aiϵA} (2) d(B, A) = max{d(bi,A), biϵB} (3) And d(ai, B) and d(bi, A) are calculated by (4) and (5) d(ai, B) = max{d(ai, bj), bjϵB} (4) d(bi, A) = max{d(bi,aj), ajϵA} (5) Distance d(ai, bj) is calculated by (6) (6) d (ai , b j ) de (ai , b j ) d0 (ai , b j ) Wherein, is the parameter to adjust the weight of the moving direction, de (ai , b j ) and d0 (ai , b j ) are calculated by (7) and (8) de (ai , b j ) ( xia xbj )2 ( yia ybj )2 (7) 8) Definiton 8: Anomaly Definition When a regional surveillance by camera, objects moving in the trajectory often made certain routes An object is called abnormal movement if it does not belong to any given trajectory groups Given the trajectory T*={t1, t2, …, tn} and the routes R={R1, R2, …, Rk} The distance from T* to R is calculated by (11) ddetect(T*, R) = mini=1 k{h(T*, Ri)} (11) With dmax is a given threshold, if ddetect(T*, R)>dmax , then T* is a abnormal trajectory 9) Definiton 9: Sub-trajectory Route segmentation is based on the velocity of moving objects Each route segmentation is called a subtrajectory The segment points are specified when the velocity is greater than the threshold The velocity of object is calculated by (12) vir vx min( i y vix vi , vix viy viy ) (12) Wherein, vix and viy are the velocity along the x-axis and the y-axis, respectively vix and viy are calculated by (13) and (14) vix xi xi (13) viy yi (14) yi Let Seg = {seg1, seg2, …, segu} be the segment points of the trajectory O (1 < segi < n, < u < n) Trajectory O is divided into u+1 segments and is shown as follows: O {t1, t2 , , tseg1 , tseg1 1, , tseg2 , , tsegu , tsegu 1, , , ,tn } The trajectories SOi {t1, t2 , , , tsegi } are called sub, trajectories Figure shows the trajectory is divided into the sub-trajectories Algorithm: Anomaly Detection Based on Sub-Trajectory of Route (ADB-STR) Input: - Umax: The maximum number of sub-trajectories - k: The number of routes - dmax: The value of threshold - {SOij }(i Figure – Segment points 10) Theorem 10 Let P be the medium route of a route P { p1 , p ,, , pseg1 , pseg1 1, , pseg2 , , psegu , psegu 1, , p n } Wherein, seg={seg1, seg2, …, segu} are the segment , tsegi } are the sub, points of P (1 < u < n) SPi {t1, t2 , , trajectories of P If the trajectopry T* is abnormal with the segment i (1≤ i ≤ u), then the trajectory T* will abnormal with all segments l (1 < l ≤ u) Proof T* is abnormal with the segment i, so ddetetct(T*, SPi) = h(T*, SPi) > dmax Suppose T* is not abnormal with the segment l (1< l ≤ u), we have ddetect(T*, SPi) < dmax Moreover, ddetect(T*, SPi) = min{h(T*,SPl), h(T*, SPi)} < dmax, hence h(T*, SPl) < h(T*, SPi) < dmax This is contrary to the hypothesis Hence, theorem completes the proof B Anomaly Detection Based on Sub-Trajectory In this paper, we popose a novel approach to detect anomaly based on sub-trajectory Our method include two phases 1) The first phase Let us denote as follows: - R = {R1,R2, …, Rk} is the routes - ri is the number of the route Ri (1 ≤ i) - O ij is the trajectory of the route Ri (1≤ i ≤ k, ≤ j ≤ ri) - P={P1, P2, …, Pk} is the medidum routes of the routes Pi is the medium route of the route Ri k );( ) (j umax ) : set of sub-trajectories - T* is the check trajectory Output: - True if detect abnormal - False if not detect abnormal j=1; Abnormal=False; While (j ≤umax and Abnormal =False) { d = min(h(T*, SOij )); if (d>dmax) then Abnormal = True; j++; } return Abnormal; III EVALUATION For experiment, we have collected 1,000 datasets from [15] Each dataset consists of 260 trajectories which includes 250 normal trajectories and 10 abnormal trajectories We divided 260 trajectories into two sets, the first set called training set contains 200 normal trajectories, the second set called testing set contains 60 trajectories which includes 50 normal trajectories and 10 abnormal trajectories Experimental procedure was divided into two phases via Mathlab R2013a The first phase was divided into steps aim to specify the threshold dmax with traiming set The second phase will detect abnormal with testing set A The First Phase - Step 1: From training set (200 normal trajectories), we divided into groups of trajectory as shown in Figure - SOij is the jth sub-trajectory of the medidum route Pi SOij {Pi( P1, P2 , , Pseg j )} This phase is carried out as follows: - Step 1: Create the trajectory group of the routes - Step 2: Calculating the medium routes by (15) Pi n i O j (t j ) nj (15) -Step 3: Specify sub-trajectories based on the medium route - Step 4: Calculating the threshold dmax by (16) dmax i i mini 1 k max j 1 ri h(O j , Pj ) Figure – groups of trajectories - Step 2: The medium routes are calculated by equation (15) as shown in Figure (16) 2) The second phase Based on theorem 10, we propose the algorithm to detect abnormal based on sub-trajectory as follows: Figure – The medium routes - Step 3: Segmenting the medium routes to determine sub-trajectories based on the changing of velocity The result is shows in table I and Figure TABLE I THE RESULT OF THE ROUTE SEGMENT Groups Segments The segment point Route 10 Route 2 Route 10 Route Route 5 REFERENCES [1] [2] [3] [4] [5] [6] [7] Figure – sub-trajectories - Step 4: Determining the threshold dmax by (14) B The Second Phase Detecting anomaly trajectory by algorithm ADB-STR with testing set The result of anomaly trajectory is shown in Figure7, the abnormal trajectories is drew by red The results of testing are fully appropriate to the experimental results of Piciarelli [15] and Laxhammar [16] However, the detection time is faster than the detection time of Piciarelli [15] and Laxhammar [16], because the proposed method mustn't process all medium routes From the result of table I, we show that the abnormal trajectory is detected at the 10th segment point, in the worst case [8] [9] [10] [11] [12] [13] [14] Figure – Abnormal trajectories [15] IV CONCLUSIONS In this paper, we have proposed a new technique to detect anomaly in video surveillance effectively In the proposed technique, the system model is built to detect anomaly based on sub-trajectory by segmenting the medium routes and modified Hausdorff distance The result of the proposed technique show that the detection time is very fast, so the technique can apply for the realtime video surveillance systems [16] S Kwak, H Byun Detection of dominant flow and abnormal events in surveillance video, Opt Eng (2011) S Wu, B.E Moore, and M Shah Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes, in: Proceedings of IEEE Computer Vision and Pattern Recognition, 2010, pp 2054–2060 R.R Sillito, and R.B Fisher Semi-supervised learning for anomalous trajectory detection, in: Proceedings of British Machine Vision Conference, 2008, pp 1035–1044 J Owens, A Hunter Application of the self-organizing map to trajectory classification, IEEE Int Workshop Vis Surveill (2000) 609–615 X Wang, X Ma, and E Grimson Unsupervised activity perception by hierarchical Bayesian models, in: Proceedings of IEEE Computer Vision and Pattern Recognition, 2007 C Piciarelli, and G.L Foresti Online-trajectory clustering for anomalous events detection in: Proceedings of Pattern Recognition Letters, 2006, pp 1835–1842 A Zaharescu, and R Wildes Anomalous behaviour detection using spatiotemporal oriented energies, subset inclusion histogram comparison and event-driven processing, in: Proceedings of European Conference on Computer Vision, 2010, pp 563–576 F Jiang, J Yuan, S A Tsaftaris, and A K Katsaggelos “Anomalous video event detection using spatiotemporal context”, Computer Vision and Image Understanding, Mar 2011, vol 115, pp 323–333 JING GAO, JIYIN SUN, KUN WU, “Image Matching Method Based on Hausdorff Distance of Neighborhood Grayscale”, Journal of Information & Computational Science 9: 10 (2012) 2855–2863 FACUNDO MÉMOLI, “On the use of Gromov-Hausdorff Distances for Shape Comparison”, Eurographics Symposium on Point-Based Graphics (2007) Christine Parent, Stefano Spaccapietra, Chiara Renso, Gennady Andrienko, Natalia Andrienko, Vania Bogorny, Maria Luisa Damiani, Aris Gkoulalas-Divanis, Jose Macedo, Nikos Pelekis, Yannis Theodoridis, and Zhixian Yan Semantic trajectories modeling and analysis ACM Comput Surv 45, 4, Article 42 (August 2013), 32 pages Yuanfeng YANG, Zhiming CUI, Jian WU, Guangming ZHANG, Xuefeng XIAN Trajectory Analysis Using Spectral Clustering and Sequene Pattern Mining Journal of Computational Information Systems 8: (2012) 2637 – 2645 Liao, L., Fox, D., and Kautz, H 2005 Location-based Activity Recognition using Relational Markov Networks In Proc of the 9th Int Conf on Artificial Intelligence, IJCAI'05 Zheng, Y., Chen, Y., Xie, X., and Ma, W.-Y 2010 Understanding transportation modes based on GPS data for Web applications, ACM Transaction on the Web, 4(1), January 2010 C Piciarelli, C Micheloni, G Foresti, “Trajectory-Based Anomalous Event Detection”, Int Proc, Int Trans on Circuits and Systems for Video technology, 2008, pp 1544-1553 Rikard Laxhammar and Goran Falkman Sequential Conformal Anomaly Detection in Trajectories based on Hausdorff Distance 14th International Conference on Information Fusion, Chicago, Illinois, USA, July 5-8, 2011  ... B Anomaly Detection Based on Sub- Trajectory In this paper, we popose a novel approach to detect anomaly based on sub- trajectory Our method include two phases 1) The first phase Let us denote as... Laxhammar and Goran Falkman Sequential Conformal Anomaly Detection in Trajectories based on Hausdorff Distance 14th International Conference on Information Fusion, Chicago, Illinois, USA, July 5-8, 2011... Andrienko, Natalia Andrienko, Vania Bogorny, Maria Luisa Damiani, Aris Gkoulalas-Divanis, Jose Macedo, Nikos Pelekis, Yannis Theodoridis, and Zhixian Yan Semantic trajectories modeling and analysis ACM