ORIGINAL ARTICLE Year : 2020  Volume : 10  Issue : 3  Page : 145157 Online handwritten signature verification and recognition based on dualtree complex wavelet packet transform Atefeh Foroozandeh^{1}, Ataollah Askari Hemmat^{2}, Hossein Rabbani^{3}, ^{1} Department of Applied Mathematics, Faculty of Sciences and Modern Technology, Graduate University of Advanced Technology, Kerman, Iran ^{2} Department of Applied Mathematics, Faculty of Mathematics and Computer; Mahani Mathematical Research Center, Shahid Bahonar University of Kerman, Kerman, Iran ^{3} Department of Biomedical Engineering, School of Advanced Technologies in Medicine; Medical Image and Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran Correspondence Address: Background: With the increasing advancement of technology, it is necessary to develop more accurate, convenient, and costeffective security systems. Handwriting signature, as one of the most popular and applicable biometrics, is widely used to register ownership in banking systems, including checks, as well as in administrative and financial applications in everyday life, all over the world. Automatic signature verification and recognition systems, especially in the case of online signatures, are potentially the most powerful and publicly accepted means for personal authentication. Methods: In this article, a novel procedure for online signature verification and recognition has been presented based on DualTree Complex Wavelet Packet Transform (DTCWPT). Results: In the presented method, threelevel decomposition of DTCWPT has been computed for three time signals of dynamic information including horizontal and vertical positions in addition to the pressure signal. Then, in order to make feature vector corresponding to each signature, log energy entropy measures have been computed for each subband of DTCWPT decomposition. Finally, to classify the query signature, three classifiers including knearest neighbor, support vector machine, and Kolmogorov–Smirnov test have been examined. Experiments have been conducted using three benchmark datasets: SVC2004, MCYT100, as two Latin online signature datasets, and NDSD as a Persian signature dataset. Conclusion: Obtained favorable experimental results, in comparison with literature, confirm the effectiveness of the presented method in both online signature verification and recognition objects.
Introduction Conventionally, authentication of a person was conducted in two types, including knowledgebased or tokenbased methods. In case of knowledgebased authentication system, the authentication is conducted based on something the user knows; the answer of the user to the secrete question(s) such as personal identification number or password.[1] However, tokenbased authentication systems work based on something the user has, such as a driver's license or identity document card.[1] However, traditionally authentication systems suffer from several treats including forgotten or stolen. These issues are addressed using biometrics. Authentication of a person using physical or behavioral characteristics is known as biometric. In the case of physical biometrics, authentication has been conducted using direct measurements of a part of human body, such as fingerprint,[2] face,[3] and iris.[4] On the other hand, behavioral biometrics use the information of an action performed by the user such as voice,[5] gait,[6] and signature.[7] Biometricbased authentication systems use many different aspects of human physiology, chemistry, or behavior. However, selection of a suitable biometric depends on several factors including: Robustness; does not have substantial changes over time Distinctively; has a great variation over different subjects Availability; all intended people have this characteristic Accessibility; easy to collect Acceptability; intended people agree to be taken from them.[8] Handwritten signature is one of the most valuable biometric traits, which has mostly been used for verification purposes in everyday life.[9] Financial and administrative institutions use handwritten signatures as legal means of verifying an individual's identity, all over the world.[9] In addition, handwritten signature has an important advantage of noninvasive and nonthreatening process by the majority of the users.[10] It should be noted that signatures written by different people are naturally different which is known as interclass variability. Furthermore, due to the impact of physical and emotional conditions of the person on its signatures, handwritten signatures of one person are not the same and this variability is known as intraclass variability.[10] Notably, in addition to high intraclass variability, some other disadvantages are included in the case of handwritten signatures: forgeries, higher error rates than other biometrics, and large temporal variation. A biometric system can be operated in two modes, namely recognition (identification) or verification. The aim of a verification system is the veracity of the person's claimed identity. However, an identification system tries to recognize the identity of the user.[8] A signature verification system deals with three kinds of forgeries: random, simple, and skilled forgeries.[11] In the case of random forgery, the forger without any information about the author's name and his/her signature reproduces a random signature. If the forger knows the author's name but does not access to any signature sample, the reproduced signature is known as a simple forgery. In the case of skilled forgery, a forger has signature samples and tries to reproduce them. Therefore, skilled forgeries are much more similar to the genuine signatures than random and simple forgeries. The focus of this paper, in the case of verification, is on the verification of skilled forgery samples. It should be noted that training in an automatic signature verification system may be writerindependent (WI) or writerdependent (WD).[12] In the first case, WI, training is conducted based on a large population of signature samples related to all persons in the dataset, whereas in the case of WD, training is done based on the signature samples of each person, separately.[12] Although WD approach achieves good results, for each user added to the system, a classifier must be conducted again which increases the complexity and cost of the system.[13] To reduce the complexity, WI approach attracts more researchers in recent years.[13],[14] In this article, we consider WI approach and a novel procedure for online handwritten signature verification and recognition is presented. Several reviews of the stateoftheart on signature identification and signature verification have been recently proposed.[15],[16],[17] Input data of the signaturebased authentication systems are in two types: offline (static) and online (dynamic). Offline handwritten signaturebased authentication systems receive only the images of signatures gathered by a camera or a scanner. However, an online signaturebased authentication system receives dynamic (temporal) information of the signing process in addition to the signature images which are gathered by a digitizing tablet or a pensensitive computer. Dynamic information includes time signals of horizontal position, x(t), vertical position, y(t), velocity, v(t), pressure, p(t), pen azimuth, z (t), and altitude, l (t).[10] There are two main categories for feature extraction, i.e., information extracted from online signatures: global and functional. The aim of a global feature extraction method is to derive feature vectors of the same length and compare them.[18] Global feature extraction method divides into two categories: In the first category, features are extracted from the totally of the signature such as average pressure, average velocity, pen tip, total signing duration, and signature height,[18],[19] while second category is dedicated to the features extracted by applying a transformation on the signature such as discrete wavelet transform (DWT) and discrete cosine transform.[20],[21] On the other hand, functional features are dedicated to the features extracted by considering time sequences describing the signing process and calculating the distances between them.[18] Functional features are in two types; in the first type, a set of reference samples related to each subject has been saved as reference set. Then, classifying is done with the comparison of the input signature with the reference set using some methods such as Dynamic Time Warping (DTW). In the second type, a model is trained using the saved signature, and then, the test signature is classified using the trained model.[18] An interesting field of research in engineering, mathematics, and bioinformatics is multiscale modeling. Multiscale modeling deals with problems with multiscale nature, i.e., having important features at multiple scales of time and/or space.[22] Notably, signals represented in time domain have important properties in the frequency domain.[23] Analysis of a time signal for its frequency content is effectively done using Fourier transform, wavelets, and other xlet transforms such as curvelet and contourlet.[23],[24] Here, a novel signature verification/recognition method using DTCWPT decomposition to obtain the frequency content of dynamic information of online signatures, has been presented. The rest of this paper is organized as follows: a brief review on DTCWPT with a literature review are presented in Subjects and Methods section. Details of the presented online signature verification method with the utilized datasets are explained in the Proposed online signature verification method section. Experimental results of signature verification and the comparison with literature are reported in the Experimental protocol and performance evaluation section. The presented online signature recognition method with the experimental results and comparison with literature are discussed in the Proposed online signature recognition method section. Finally, we conclude our work and outline some future work directions in the Conclusion and Future Works section. Subjects and Methods This section is dedicated to a brief review on basic concept used in the presented method, i.e., DTCWPT, with a literature review on the related signature verification methods. A brief review on dualtree complex wavelet packet transform Fourier transform is one of the most popular tools used in the field of multiscale modeling. In the Fourier transform, infinitely oscillating sinusoidal basis functions are used to represent the input signal. Some of the limitations of Fourier transform have been modified using wavelet transform.[20] Indeed, oscillating sinusoidal basis functions used in the Fourier transform are replaced with wavelets as basic functions which are locally oscillating in the DWT.[20] Wavelets are stretched and shifted versions of a realvalued bandpass wavelet Ψ(t). To form orthonormal basis for the space of onedimensional realvalued continuoustime signals, wavelets are combined with shifted versions of a realvalued lowpass scaling function φ(t). Therefore, any finite energy signal x(t) can be decomposed in terms of wavelets and scaling function as follows: [INLINE:1] Where φ(.) and Ψ(.) are computed using lowpass and highpass filters, h0(n) and h1(n), respectively, as follows: [INLINE:2] [INLINE:3] Further, the scaling coefficients, c(n), and wavelet coefficients, d(j, n), are computed using the inner product, as follows: [INLINE:4] [INLINE:5] In DWT, only low frequency band is decomposed at each level of decomposition. The procedure of a twolevel decomposition of DWT has been shown in [Figure 1]a. DWT has some advantages including good compression, perfect reconstruction, no redundancy, and very low computation.[25] However, DWT suffers from four deficiencies, including oscillations, shift variance, aliasing, and lack of directionality.[26]{Figure 1} An octaveband analysis of the frequency domain is provided by DWT which might not be optimal for a given signal. Finding an optimal representation is possible with discrete wavelet packet transform (DWPT).[27],[28] In DWT, only low frequency band is decomposed at each level, but in DWPT, decomposition is conducted for both low and high frequency bands. It should be noted that DWPT is also shiftvarying and suffers from lack of directionality in twodimensions (2D) and higher dimensions, like DWT. These deficiencies are approximately addressed by dualtree complex wavelet transform (DTCWT).[28] The structure of DTCWT is composed of two DWTs; coefficients of the first and second DWTs are combined as real and imaginary parts of DTCWT. Suppose that the first and second wavelets are denoted by Ψ(t) and Ψ'(t), respectively. These wavelets are defined similar to Eq. 2 and Eq. 3 using {h0(n), h1(n)} and {h0'(n), h1'(n)}, for Ψ(t) and Ψ'(t), respectively. [Figure 1]b shows the structure of twolevel decomposition of DTCWT. It should be noted that Hilbert transform of a function U(t), is computed as follows: [INLINE:6] To improve the shiftability of DWT, in DTCWT, the second wavelet, Ψ'(t), is the Hilbert transform of the first wavelet,Ψ(t), i.e. Ψ' = H{Ψ(t)}. Advantages of DTCWT include low computation, limited redundancy, perfect reconstruction, good directional selectivity, good shift invariance, and analyticity in 1D.[25] As the DWPT extending the DWT, DTCWPT extends the DTCWT.[28] DTCWPT is obtained by iterating two perfect reconstruction filter banks on the lowpass and highpass outputs.[28] The design of these filters is such that the response of each branch of the second wavelet packet filter bank is the discrete Hilbert transform of the corresponding branch of the first wavelet packet filter bank. Therefore, each subband of the DTCWPT will be approximately analytic.[28] This is true when the filters used in the second wavelet packet of DTCWPT are the same with the filters used in the first wavelet packet of it. Notably, similar to DTCWT, approximately shiftinvariance and good directional selectivity are provided with DTCWPT. [Figure 1]c shows the structure of twolevel decomposition of DTCWPT. As shown in [Figure 1]c, for klevel decomposition of DWT, DTCWT, and DTCWPT of ID signal, the number of output subbands is k + 1, 2(k + 1), and 2k + 1, respectively. Literature review Traditional wavelets and extensions of wavelets, such as DWPT and DTCWT, have been widely used in the structure of the presented signaturebased authentication systems, until today. In the following, some of the presented online signature verification methods based on wavelet or similar transforms have been briefly reviewed. An online signature verification system based on DWT features and neural network classification has been presented in the study by Maged and Fahmy.[21] Threelevel decomposition of DWPT has been implemented on time signals, including horizontal and vertical position, pressure, pen azimuth, and pen altitude, in the study by Wang et al.[29] Then, different combinations of these decompositions have been considered as extracted features. Finally, decision about genuine or forgery of the query signature has been made using the Euclidean distance. Continuous wavelet transform has been used to obtain the frequency information of the speed signals in.[30] The grayscale spectrograms created by wavelet transforms have been used to train support vector machine (SVM) network. In the study by Chang et al.,[31] five time signals including horizontal and vertical position, pressure, pen azimuth, and altitude have been considered for each signature. Then, different wavelets such as Haar, Daubechies, Symlet, and Coiflet have been used to decompose the time signal up to fivelevel decomposition. Final decision about the query signature has been conducted using a threshold on the computed distance measures. An improved waveletbased online signature verification scheme has been presented in the study by Nilchiyan and Yusof.[32] In this work, five time signals including horizontal and vertical position, pressure, pen azimuth, and altitude have been discussed, and threelevel decomposition of them using Haar and Daubechies wavelets have been computed for feature extraction. Classification has been done using the multiperceptron neural network with one intermediary layer. It should be noted that, in comparison with signature verification, there are fewer works conducted for online signature recognition, until today. To the best of our knowledge, there is no online signature recognition work based on wavelet or other similar transforms. However, there are few other procedures for online signature recognition, which have been briefly reviewed in the following. An online signature slant identification was presented in the study by Shamsuddin and Mohamed.[33] Horizontal and vertical positions of the signing process have been considered and filtered them. Then, the angle and degree of the signature have been computed and the signature has been classified into its slant category using a slant algorithm. This work presented an accuracy of 80% on a private dataset. In the study by Mohamed et al.,[34] a baseline extraction algorithm has been used for online signature recognition based on vector rules. Direction, slant, baseline, pressure, speed, and numbers of pen ups and downs have been used as the main features. An algorithm to extract baseline from signature has been used and obtained the accuracy of 90% on their created dataset. An online signature recognition based on some global features such as standard deviation and maximum of the velocity and the acceleration in both x and y directions has been presented in the study by AlMayyan et al.[35] A rough set classifier has been used and the accuracy of 100% for recognition rate has been obtained on its own created dataset composed of 2160 signatures from 108 subjects. Concepts of graph theory have been used for online signature identification in the study byFotak et al.[36] A fast classification using graph norm and comparison between each signature graph concepts value with the saved values in the dataset has been conducted to obtain an identification accuracy of 94.25% on their own dataset. However, to the best of our knowledge, DTCWPT has not been used in the structure of online signature recognition and verification systems, until today. In this paper, a writerindependent online handwritten signature recognition and verification system is presented based on DTCWPT decomposition of time signals containing dynamic information of each signature. Proposed online signature verification method In this section, the novel presented method for online handwritten signature verification based on DTCWPT has been described in detail. The block diagram of the presented signature verification method is shown in [Figure 2].{Figure 2} Online handwritten signature datasets The performance of the presented online signature verification method has been evaluated using three benchmark available datasets, including SVC2004[37] and Ministerio De Ciencia Y Tecnologia (MCYT)100,[38] as two benchmark online handwritten Latin signature datasets, and Noshirvani Dynamic Signature Dataset (NDSD),[39] as an online handwritten Persian signature dataset. The statistics of these datasets are displayed in [Table 1].{Table 1} SVC2004 (https://www.cse.ust.hk/svc2004/) is an online handwritten Latin signature dataset established for the First International Signature Verification Competition (SVC2004).[37] WACOM INTUOS tablet was used for saving the dynamic information of the signing process. SVC2004 dataset provided two different signature databases, namely Task 1 and Task 2. The main SVC2004 dataset includes 100 contributors: 40 persons included in Task 1 and other 60 persons included in Task 2 signature dataset. Task 2, which has been considered in this study, contains 40 signers with 20 genuine signatures and 20 skilled forgeries per signer. Genuine signatures were signed in 2 weeks and skilled forgeries were signed with four other contributors for each contributor's signature. Dynamic information saved in Task 2 used in the presented method is shown in [Table 1]. MCYT (http://atvs.ii.uam.es/atvs/mcyt100s.html) dataset composed of a fingerprint and an online Latin signature dataset.[38] Here, the public subset of MCYT contains signatures of 100 signers (MCYT100) has been used. Dynamic information shown in [Table 1] was saved using WACOM pen tablet, model INTUOS A6 USB with a capture area of size 127 × 97.[38] There are 25 genuine signatures and 25 skilled forgeries for each individual enrolled in the dataset. Forgeries were produced by five subsequent target users trying to copy the statistic images of the signature in 10 times. NDSD (https://sites.google.com/site/meyahyatabar/ND SD % 20 Data.rar?attredirects = 0 and d = 1) is an online handwritten Persian signature dataset. WACOM INTUOS4 digitizing tablet was used to save dynamic information of signing process.[39] Signers signed on a plate of size 96 × 129 and the information was saved every 10 ms. These dynamic informations are shown in [Table 1]. Signers were in the range of 18–40 years old and signed 65 signatures in two different times with more than 3 days' interval. Further, 15 professional forgers forged 40 samples from 65 genuine signatures for each person.[39] As an example, one sample of online signature image from MCYT100 signature dataset with the corresponding signals of horizontal position (xcoordinate), vertical position (ycoordinate), and pressure is shown in [Figure 3].{Figure 3} Preprocessing Among dynamic information describing the signing process available in the used datasets [Table 1], only three time signals including xcoordinate (x(t)), ycoordinate (y(t)), and pressure (p(t)) have been considered in the presented online signature verification method. Due to the intraclass variability, time signals corresponding to each signer have different length and size. Hence, some preprocessing tasks including length normalization, rotation normalization, and size normalization have been conducted on the mentioned time signals. Length normalization has been conducted to equalize the number of points of each signal. This is done using resampling of the signal. Further, in the process of gathering signatures, handwritten signatures are recorded in different angles, and so, rotation normalization is necessary to reduce intraclass variability. Rotation normalization has been conducted similar to.[40] In addition, since different dynamic ranges and to minimize this range and set the value changes in a specified interval, Minmax normalization has been conducted to set the values in (0,100) using Eq. 7, and then, they become standard using Eq. 8, as follows:[35] [INLINE:7] [INLINE:8] Where f(t) is the values of a time signal and fMin and fMax are the minimum and maximum of the signal in hand, respectively. Notably, μ and σ are the average and standard deviation of the signal. Proposed online signature verification method After conducting the preprocessing tasks on the time signals including xcoordinate, ycoordinate, and pressure, feature extraction step has been conducted. Due to the intraclass and interclass variabilities, an effective signature verification system should be considered features with the ability to minimize intraclass variability and maximize interclass variability. The best way to do this is using the multiscale modeling.[21],[29] In this article, a global feature extraction approach using DTCWPT as multiscale modeling tool has been considered. To extract features from time signals, threelevel decomposition of DTCWPT has been performed on each time signal. The output subbands include the local characteristics of the original signal in threelevel depth of decomposition. However, the number of these coefficients is so huge to directly use as the classifier input.[41] Therefore, some features should be derived from these coefficients to enhance the performance of the classifier. To extract features, entropy concept has been used. Entropy is a common concept in many fields of research, mainly in signal processing,[41] and it is one of the widely used tools to extract features. Entropy measures the uncertainty of the information included in the signals.[41] There are several types of entropy including Shannon, Renyi, Tsallis, and log energy.[41] Shannon entropy of a time signal as a wellknown entropy is a measure of its average uncertainty and computed as follows: [INLINE:9] Where fi is the value of the time signal f(t) at time ti. Shannon entropy has been used in the structure of the signature verification system, until today.[42],[43] In this article, the log energy entropy (LEE) has been used as follows: [INLINE:10] To extract features, LEE for all subbands obtained from threelevel decomposition of DTCWPT has been computed. DTCWPT is composed of two wavelet packets and each of them yields to 16 (=23+1) subbands. Hence, totally, 32 (=16 × 2) subbands have obtained with threelevel decomposition of DTCWPT. LEE of all of these subbands has been computed and concatenated to each other to make a feature vector corresponding to the original signature. To classify the signatures in the feature space, three approaches have been tested for classification: knearest neighbor (kNN), SVM, and KolmogorovSmirnov (KS) test as a statistical classification approach. kNN is a nonparametric classification technique, i.e., it does not make any assumptions on the underlying data distribution.[44] kNN works based on feature similarity and its output is a class membership, a discrete value to predict a class by a majority vote of its neighbors, and being assigned to the class most common among its kNNs (https://blog.usejournal.com/aquickintroductiontoknearestneighborsalgorithm62214cea29c7).[44] SVM classification as a supervised learning algorithm is one of the most popular classifiers used in machine learning, pattern recognition, and signal processing.[45] The idea behind the SVM classification is finding a hyperplane with best separation between the data in the feature space.[46] In the case of nonlinear decision function, a nonlinear transformation is used to map the data from the input feature space into a highdimensional feature space. According to the Cover's theorem and for nonlinear transformation and highdimensional space, the patterns are linearly separable with high probability in the new feature space.[45] This nonlinear transformation is performed using kernel functions. There are different types of kernel including linear, polynomial, and radial basis function (RBF). RBF is one of the most popular kernel functions whose value depends on the distance from the origin or from some point. RBF kernel for two feature vectors X and X' is defined as follows: [INLINE:11] Where ‖ ‖2 is the squared Euclidean distance and σ is a free parameter. Equivalently, with [INSIDE:1], we have: [INLINE:12] KS test is a nonparametric hypothesis test which is widely used for comparing two given distance distributions of a single independent variable.[47] The KS static for comparing two distributions P0 and P is computed as follows:[47] [INLINE:13] Where Fm(.) and Gn (x) are empirical cumulative distribution functions (c.d.f.) of two given distance distributions P0 and P, respectively. The P value for this test of hypothesis is computed as follows: [INLINE:14] Where H(.) is the c.d.f. of KS distribution with the following definition:[47] [INLINE:15] KS test for two samples tests the null hypothesis [INSIDE:2], i.e., two samples come from the same distribution and accept the null hypothesis, if the computed P value is greater than a significance level (α), else the null hypothesis is rejected. It should be noted that in testing hypothesis, two types of error can occur: type I and type II.[47] Type I error occurs if the null hypothesis is rejected when it is true, and type II error occurs if the null hypothesis is accepted when it is false. The probability of committing a type I error in a decision rule is called the significance level (α). Significance level is usually considered as 5%, or 1%, and the null hypothesis is accepted with confidence level 95% (= 1 − α, for α = 5%).[47] Experimental protocol and performance evaluation In this section, details of the conducted experiments for the presented online signature verification in addition to the obtained experimental results have been described. At the end of this section, the presented method has been compared with literature. Experimental protocol for the proposed online signature verification method Three time signals xcoordinate, ycoordinate, and pressure have been considered as dynamic information corresponding to each signature in the dataset. Preprocessing tasks including length normalization, rotation normalization, and size normalization have been conducted on the mentioned time signals. Feature extraction has been conducted based on DTCWPT on the preprocessed data. DTCWPT has two separate wavelet packets. To get features from the different scales of the signals, threelevel decomposition of DTCWPT has been performed on the signals. Therefore, 16 subbands have been obtained for each signal at each wavelet packet. Notably, the length of these subbands is dependent to the length of the input signal which is different between datasets and also their users. The number of these coefficients is so huge to directly use as the classifier input, and therefore, LEE of each DTCWPT subband has been computed as features. These features have been computed for 32 all subbands from both first and second wavelet packets of the threelevel decomposition of a DTCWPT for every signature in the dataset and concatenated in a vector as a feature vector of the signature. Therefore, for each time signal, 32 entropy measures have been computed as features per time signal. Totally, with considering three mentioned time signals, the final feature vector of length 96 (= 3 × 32) has been considered for each signature in the dataset. These feature vectors have been computed for every genuine signatures and skilled forgery samples in the dataset and then fed into the classification step. Three classification techniques including kNN, SVM, and KS test have been considered to make decision for acceptance/rejection of the test signature. The parameters of these classification techniques have been empirically set as follows; kNN with k = 3 neighbors and SVM with RBF kernel and regularization parameter C = 1000. Regularization parameter trades off the correct classification of training examples against maximization of the decision function's margin (https://www.mathworks.com/help/stats/fitcsvm.html). These values of the parameters work well for the problem; however, we noted that they could be optimized which is not explored in this work. Here, an online handwritten signature verification with the WI approach has been presented. It should be noted that there are no skilled forgeries for each user enrolled to the system in real situations. Hence, to applicability of the presented system, training of the classifier only conducted with genuine signatures available in the dataset. To train the classifier and to avoid overfitting, feature vectors of genuine signatures have been divided into three sets: training, validation, and testing with 60%, 20%, and 20% of data, respectively. Then, the feature vectors of all of the forgery samples have been added into testing set and the final results have been reported using threefold cross validation. In the case of KS test, making decision for acceptance/rejection of the query signature is formulated as a testing hypothesis, i.e., decision between two existed status: the query signature belongs to the person having claimed his/her identity (null hypothesis) or it is a forged sample (alternative hypothesis) (https://machinelearningmastery.com/statisticalhypothesistests/). In other words, KS test is a decision between the following hypotheses: [INLINE:16] Where P0 is considered as the distribution of distances between training samples of true person, and P is considered as the distribution of distances between the query signature with all true saved samples of the person having claimed. Two distributions P0 and P include pairwise distances between feature vectors of the signatures. These distances have been computed using Euclidean distance, in this article. Naturally, two distance distributions P0 and P are slightly different. However, in signature verification process, the goal is answering to the question that whether existed difference is significant for failing the null hypothesis (rejection the query signature) or not. To do this, KS test with α = 5% has been considered in our experiments. The signature in question has been claimed as a genuine signature (accept the null hypothesis) or a skilled forgery sample (reject the null hypothesis) with confidence level of 95% (= 1 − α). Obtained experimental results are provided in the next section. Performance evaluation of the proposed online signature verification method The performance of the presented signature verification method has been evaluated using three wellknown criteria: false rejection rate (FRR), false acceptance rate (FAR), and equal error rate (EER). FRR is the fraction of the genuine signatures which have been falsely rejected, FAR is the fraction of skilled forgeries falsely accepted, and EER is the error rate when FRR = FAR. The presented signature verification method has been implemented using a system with Intel Core i77500U for CPU and 12 GB of RAM. Here, the MATLAB implementation of DTCWPT has been used with threelevel decomposition (https://ilkerbayram.github.io/dtcwpt/),[28] and the implementation of the presented method has been conducted using the programming language MATLAB R2017a (Natick, Massachusetts, USA). Online signature verification results and comparison with literature Several experiments have been conducted using three datasets of two scripts, including SVC2004 and MCYT100, as two publicly available benchmark Latin datasets, and NDSD as a benchmark Persian dataset. Obtained experimental results are shown in [Table 2] for three classification techniques: kNN, SVM, and KS test. As shown in [Table 2], among three classification techniques, the best results in all three datasets have been obtained using KS test for decisionmaking about genuine/forgery of the query signature. After KS test, SVM with RBF kernel and kNN classification obtained the best results, respectively.{Table 2} To better intuition about how to separate the genuine signatures from forgery samples in the space of their feature vectors, tdistributed stochastic neighbor embedding (tSNE) (https://lvdmaaten.github.io/tsne/) algorithm has been used. tSNE algorithm is a successful method which has been presented for dimensionality reduction introduced by van der Maaten and Hinton in 2008.[48] tSNE technique is used for visualizing highdimensional data into a lowdimensional space of two or three. Here, tSNE algorithm has been used to visualize the feature vectors from feature space of 96dimensional into 2D space. [Figure 4] shows 2D projections of the feature vectors corresponding to the 65 genuine signatures and 40 skilled forgeries related to one user in NDSD dataset. Further, [Figure 5] shows 65 genuine signatures and 40 skilled forgeries related to four different users in NDSD dataset. As shown in these [Figure 4] and [Figure 5], the extracted features can make good separation between genuine signatures and skilled forgeries, which verifies the favorable signature verification results.{Figure 4}{Figure 5} To compare the obtained experimental results with literature, several prominent online handwritten signature verification methods have been selected and compared using [Table 3]. [Table 3] compares the presented signature verification method with some prominent methods implemented on three datasets: SVC2004, MCYT100, and NDSD. As shown in [Table 3], in the case of MCYTY100 and NDSD datasets, the presented signature verification method outperformed the stateoftheart. In the case of SVC2004, the stateoftheart is dedicated to the presented method by Fayyaz et al.[49] It should be noted that Fayyaz et al.[49] used taught learning to learn feature from 17500 signature images of ATVS dataset,[50],[51] and verification process has been conducted on SVC2004 to get 0.83% for EER. In comparison with this method, our presented method has lower computational cost which using handcrafted features, instead of learning features.[49] However, [Table 3] shows that in comparison with other prominent works, our method also obtained promising results in the case of SVC2004.{Table 3} Proposed online signature recognition method In this section, the proposed procedure for online signature recognition with the corresponding experiments, obtained results, and comparison with literature has been presented. Proposed online signature recognition method In this section, an online handwritten signature recognition has been presented. Similar to the presented signature verification method, three time signals xcoordinate, ycoordinate, and pressure have been considered and the preprocessing tasks have been conducted on them for each signature in dataset. Further, to extract features, a threelevel decomposition of DTCWPT has been performed on the mentioned three time signals. Then, entropy measures using LEE have been computed for each of the obtained subbands of DTCWPT decomposition. These entropies have been concatenated to each other to make the feature vector related to each signature in dataset. Finally, to recognize the class of the query signature in the presented signature recognition method, two classifiers including kNN and SVM have been considered. Online signature recognition results and comparison with literature Three benchmark online signature datasets, including SVC2004 and MCYT100, as two Latin datasets, and NDSD, as a Persian dataset, have been considered for experiments. Notably, in the case of signature recognition, only genuine signatures from each dataset have been used. The presented signature recognition method has been evaluated using true recognition rate, i.e., the percentage of truly classified signatures on the datasets. To classify the query signature, two classifiers with the following parameters have been used: kNN with k = 3 and SVM with RBF kernel and the regularization parameter C = 1000. These parameters have been empirically set for our experiments. Notably, in the case of SVM classifier, in addition to RBF kernel, polynomial kernel and linear kernel have been used to evaluate the presented signature recognition method. However, the best results have been obtained by RBF kernel of SVM classifier. To avoid overfitting, final signature recognition results have been reported using threefold cross validation. To the best of our knowledge, KS test has not been used for recognition purpose, until today. This also has not been used here. Authors believe that this is because of the high computations and inaccurate decisions that may occurred with KS test in the recognition decisions. This is explained here. To recognize the class of the query signature, the comparison should be done with all classes of people registered in the system. Suppose that there exist M classes of writers. Therefore, computed pairwise distances between signatures in each class lead to make M distributions of distances. In addition, computed distances between the query signature and the signatures in each class make M other distributions. KS static should be computed between distribution of one class as P0, and distribution of the query signature and that class, as P1. The computed KS static is compared to the considered P value. Then, between all of the acceptable KS statics, the class of the query signature has been selected as the class with the maximum KS static. Therefore, as explained, the computations are increased and make a final decision can be confusing when some equal KS statics have been computed. The presented signature recognition method has been implemented using a system with Intel Core i77500U for CPU and 12 GB of RAM. Here, the MATLAB implementation of DTCWPT has been used with threelevel decomposition,[28] and the implementation of the presented method has been conducted using the programming language MATLAB R2017a. [Table 4] shows the obtained signature recognition results. As shown in [Table 4], the best recognition rate has been obtained using SVM with RBF kernel on three considered datasets.{Table 4} To visualize signature recognition results, tSNE algorithm has been used to better intuition about how to separation of signatures from different classes in the space of the feature vectors. Here, tSNE algorithm has been used to visualize the feature vectors from feature space of 96 dimensions into 2D space. As an example, eight users from NDSD dataset have been considered and 2D projections of the feature vectors corresponding to 30 signatures of each user have been shown using tSNE algorithm, in [Figure 6]. As shown in [Figure 6], there is a geed separation between genuine signatures from eight different classes in their feature vectors, which leads to obtain favorable online signature recognition results shown in [Table 4].{Figure 6} To compare the signature recognition results with literature, some of the prominent presented signature recognition methods have been considered. It should be noted that, in comparison with literature on signature verification methods, fewer methods have been presented for online signature recognition. To the best of our knowledge, all of these methods have been evaluated on a private dataset and the comparison is not fair. However, their obtained true recognition rates and our obtained recognition rate are shown in [Table 5]. As shown in [Table 5], the previous works presented good results and the method in the study by AlMayyan et al.[35] obtained the accurate recognition rate on its own created dataset composed of 2160 signature from 108 users. However, obtained signature recognition results on three benchmark dataset verify the suitably and effectiveness of the presented signature recognition method.{Table 5} Conclusion and Future Works In this paper, a novel approach for online handwritten signature verification and recognition has been presented. Dynamic information including three time signals xcoordinate, ycoordinate, and pressure have been considered to extract features. DTCWPT has been used as a multiscale transform with threelevel decomposition of the time signals. Using this decomposition, 32 subbands have been obtained for both of two wavelet packets of DTCWPT. Then, instead of the huge number of coefficients of DTCWPT subbands, entropy measures of each subband have been computed as feature corresponding to that subband. In the presented method, LEE has been used and the computed features of each subband have been concatenated to make the feature vector related to each signature in dataset. Three classifiers KNN with k = 3, SVM with RBF kernel, and KS test with confidence level of 95% have been used to final decision about the query signature to be a genuine signature or considered as a forgery sample. The performance of the method has been evaluated on three publicly available benchmark datasets: SVC2004 and MCYT100 as two Latin datasets and NDSD as a Persian dataset. Experimental results show that the best classification has been conducted using KS test on three mentioned datasets. Comparison with literature verifies that the presented method using DTCWPT and LEE outperformed the stateoftheart on MCYT100 and NDSD datasets and also obtained really promising results on SVC2004 dataset. Similar to the presented signature verification method, an online signature recognition method has been presented. After conducting the preprocessing step on three time signals: xcoordinate, ycoordinate, and pressure, their corresponding feature vectors have been computed using threelevel decomposition of DTCWPT and LEE measures. The classification for recognition of the user's class has been conducted using kNN and SVM classifiers and the final reported results have been obtained by threefold cross validation. Obtained results show favorable accuracy around 99% on three benchmark datasets which verifies the effectiveness of the presented method. A possible future work for improving the performance of the presented online signature verification and recognition systems is using other multiscale transforms, other types of entropy measures, and other statistical test in the structure of the systems. In addition, the performance of DTCWPT will be evaluated in the case of offline handwritten signature verification and recognition, by us.[56] Acknowledgment We would like to thank the reviewers for their valuable suggestions and comments during the revision process. Financial support and sponsorship None. Conflicts of interest There are no conflicts of interest. BIOGRAPHIES [INLINE:17] Atefeh Foroozandeh, received her B.Sc. and M.Sc. degrees in Applied Mathematics from Ferdowsi University of Mashhad, Iran, in 2007 and 2010, respectively. She is now a PhD student of Applied Mathematics, Wavelet Analysis in Graduate University of Advanced Technology, Kerman, Iran. Her current research interests include handwritten signature recognition and verification (online and offline), deep learning, and the application of Xlets, sparse transforms and fractal theory in the field of signal and image processing. Email: [email protected] [INLINE:18] Ataollah Askari Hemmatreceived received his B.Sc. degree in Mathematics and his M.Sc. degree in Analysis from Shahid Bahonar University of Kerman, Kerman , Iran, in 1987 and 1990, respectively. He received his Ph.D. degree in Analysis (Wavelets) from Shiraz University, Shiraz, Iran in 2000. He is now a full professor in Department of Applied Mathematics, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman. His field of specialization include Functional Analysis, Fourier Analysis, Wavelet, Frame Theory, and Harmonic Analysis. His research interests are Analysis, wavelets, and application of wavelets in solving differential equations. Email: [email protected] [INLINE:19] Hossein Rabbani received the B.Sc. degree in Electrical Engineering (Communications) from Isfahan University of Technology, Isfahan, Iran, in 2000 with the highest honors, and the M.Sc. and Ph.D. degrees in Bioelectrical Engineering in 2002 and 2008, respectively, from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran. In 2007 he was with the Department of Electrical and Computer Engineering, Queen's University, Kingston, ON, Canada, as a Visiting Researcher, in 2011 with the University of Iowa, IA, United States, as a Postdoctoral Research Scholar, and in 20132014 with Duke University Eye Center as a Postdoctoral Fellow. He is now a Professor in Biomedical Engineering Department and Medical Image & Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran. Dr Rabbani is a Senior Member of IEEE, IEEE Signal Processing Society and IEEE Engineering in Medicine and Biology Society. His main research interests are medical image analysis and modeling, statistical (multidimensional) signal processing, sparse transforms, and image restoration, which more than 110 papers and book chapters have been published by him as an author or coauthor in these areas. Email: [email protected] References


