臺灣博碩士論文加值系統

Structural analysis of video is an essential step to automatic video content
analysis. In a general video structure, shot is the most basic unit, and it must
be determined before any multimedia content-based retrieval applications.
However, despite the rich research efforts in software shot detection system,
there is a lack of shot detection system that is designed in hardware. In
this thesis, we proposed a shot boundary detection system using hardwarebased
Support-Vector-Machine(SVM). Our system can detect both cut shots
and gradual shots. We optimized the feature extraction process based on
global color histogram with a pipelined architecture to save memory and
increase the overall speed. The class imbalance problem in shot detection is
solved by using random pseudo-sampling at the SVM training stage. The
digital hardware SVM is designed using a fully-parallel pipelined architecture
and is highly configurable on vector dimensions. Our data wordlength only
used 4 bits signed integer plus one bit decimal in fixed number, while the
detection accuracy is competitive comparing with floating point software.
Our SVM classifier presented here can run a speed up to 251.62MHZ on
Xilinx Virtex IV XC4VSX35 FPGA, and the video processing on our shot
detection system can achieve 128 fps on PC. That makes our system met the
real-time constraints, and detect shots in a continous video stream such as
live-TV news or sports game.

1 Introduction 1
1.1 Overview or shot boundary detection System . . . . . . . . . . 1
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Our Contribution . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Thesis Organization . . . . . . . . . . . . . . . . . . . . . . . . 9
2 Shot Boundary Detecton System and SVM 10
2.1 Introduction of the Shot Boundary Detection System . . . . . 11
2.1.1 Features Extraction . . . . . . . . . . . . . . . . . . . . 12
2.1.1.1 General Features ExtractionMethods . . . . 13
2.1.1.2 Temporal Kernel Correlation Filtering . . . . 15
2.2 Support Vector Machine . . . . . . . . . . . . . . . . . . . . . 17
2.2.1 Brief Introduction . . . . . . . . . . . . . . . . . . . . . 17
2.2.2 Shot boundary classification using SVM . . . . . . . . 19
2.2.3 Hardware Implementation of SVM . . . . . . . . . . . 20
2.2.4 Hardware Shot Detector . . . . . . . . . . . . . . . . . 21
3 System and Hardware Architecture 22
3.1 Feature Extraction in Shot Boundart Detection . . . . . . . . 23
3.1.1 Visual Representation of Video . . . . . . . . . . . . . 23
3.1.2 Continuity FeatureMeasurement . . . . . . . . . . . . 24
3.1.3 A Pipelined Temporal Kernel Correlation Architecture 25
3.2 Support Vector Machine Architecture . . . . . . . . . . . . . . 26
3.2.1 Proposed Hardware Architecture . . . . . . . . . . . . 28
3.2.1.1 Hardware design platform . . . . . . . . . . . 28
3.2.1.2 Class Imbalance in Shot Boundary Detection 29
3.2.1.3 A Fully-Parallel Pipelined SVM . . . . . . . . 30
3.2.2 Post-processing Refinement of Shot Detection System . 33
4 Experimental Result 35
4.1 EvaluationMetrics . . . . . . . . . . . . . . . . . . . . . . . . 35
4.2 Training using RandomPseudo-Negative sampling . . . . . . . 36
4.3 Hardware Synthesizing Results . . . . . . . . . . . . . . . . . . 38
4.3.1 Result compares with other SVMhardwares . . . . . . 38
4.3.2 Result compares with other hardware shot detector . . 39
4.4 Wordlength Quantization . . . . . . . . . . . . . . . . . . . . . 40
4.4.1 Discussion on Effects of Quantization in Shot Detection 42
4.5 Performance of the Proposed Shot Detection System . . . . . 44
4.5.1 Realtime Shot Detection on Various Video Types . . . 45
5 Conclusion 48
5.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.2 FutureWork . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

[1] H.J. Zhang, Low, Smoliar. “Automatic partitioning of full-motion
video”, Multimedia Systems 1, 1993
[2] Koprinska, Carrato. “Temporal video segmentation: A survey”, Signal
Processing: Image Communication,2001
[3] Faisal M.Khan et.al. “Hardware-Based Support Vector Machine Classification
in Logarithmic Number Systems” Circuits and Systems, 2005,
ISCAS, IEEE Symposium on
[4] J. Yuan, et.al. “ A Formal Study of Shot Boundary Detection”, Circuit
and Systems For Video Technology, 2007, IEEE Transaction on
[5] V.Vapnik. “The National of Statistical Learning Theory”, 1st ed., New
York: Springer-Verlag, 1995
50
[6] Chih-Chung Chang and Chih-Jen Lin, LIBSVM : a library
for support vector machines, 2001. Software available at
http://www.csie.ntu.edu.tw/˜cjlin/libsvm
[7] M. Cooper and J. Foote. Scene Boundary Detection Via Video Self-
Similarity Analysis. Image Processing,2001,Proc. IEEE Intl. Conf. on
[8] M.Cooper, et.al. “Video Segmentation via Temporal Pattern Classification”,
Transaction on Multimedia, 2007, IEEE Transaction on
[9] D.Anguita, A.Boni, et.al.“A Digital Architecture for Support Vector
Machines: Theory, Algorithms, and FPGA Implementation,” Neural
Networks, 2003, IEEE Transaction on
[10] A. Hanjalic. “Shot boundary detection: unraveled and resolved?,” Circuits
Syst. Video Technol, 2002, IEEE Transaction on
[11] R.Genov, et.al.“Kerneltron: Support Vector Machine in Silicon,”, Neural
Networks,vol.14, no.5, pp.1426-1434, 2003, IEEE Transactions on
[12] R.Herbich. Learning Kernel Classifiers:Theory and Algorithms. Cambridge:
The MIT Press, 2002.
[13] A. Amir, et al.,“IBM Research TRECVID-2003 Video Retrieval System”
TRECVID Workshop, 2003.
51
[14] J.Yuan, et al., “A unified shot boundary detection framework based on
graph partition model,” Multimedia, 2005, ACM proc.
[15] Ngo, “A robust dissolve detector by support vector machine,” Multimedia,
2003, ACM proc.
[16] T.Chua, et al.,“An unified framework for shot boundary detection via
active learning,” Proc. ICASSP, 2003
[17] Feng, et.al., “A new general framework for shot boundary detection and
key-frame extraction,” Int. Workshop Multimedia Inf. Retrieval, 2005,
ACM SIGMM
[18] Burges, “A tutorial on support vector machines for pattern recognition,”
Data mining know. Discov.,, 1998
[19] Apostol Natsev, Milind Naphade, Jelena Tesic,“Learning the Semantics
of Multimedia Queries and Concepts from a Small Number of Examples,”
2005, ACM Multimedia
[20] J.S. Boreczky, L.A. Rowe, ”Comparison of video shot boundary detection
techniques,” Proc of SPIE- Storage and Retrieval for Still Image
and Video Databases IV, Vol. 2670, San Diego, 1996
[21] A real-time shot cut detector: Hardware implementation, L. Boussaid,
et.al, “Standard & Interfaces”, 2007, ACM