

ORIGINAL ARTICLE 

Year : 2020  Volume
: 10
 Issue : 3  Page : 145157 

Online handwritten signature verification and recognition based on dualtree complex wavelet packet transform
Atefeh Foroozandeh^{1}, Ataollah Askari Hemmat^{2}, Hossein Rabbani^{3}
^{1} Department of Applied Mathematics, Faculty of Sciences and Modern Technology, Graduate University of Advanced Technology, Kerman, Iran ^{2} Department of Applied Mathematics, Faculty of Mathematics and Computer; Mahani Mathematical Research Center, Shahid Bahonar University of Kerman, Kerman, Iran ^{3} Department of Biomedical Engineering, School of Advanced Technologies in Medicine; Medical Image and Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran
Date of Submission  04Sep2019 
Date of Decision  25Sep2019 
Date of Acceptance  23Nov2019 
Date of Web Publication  03Jul2020 
Correspondence Address: Prof. Ataollah Askari Hemmat Department of Applied Mathematics, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman, Kerman Iran
Source of Support: None, Conflict of Interest: None
DOI: 10.4103/jmss.JMSS_44_19
Background: With the increasing advancement of technology, it is necessary to develop more accurate, convenient, and costeffective security systems. Handwriting signature, as one of the most popular and applicable biometrics, is widely used to register ownership in banking systems, including checks, as well as in administrative and financial applications in everyday life, all over the world. Automatic signature verification and recognition systems, especially in the case of online signatures, are potentially the most powerful and publicly accepted means for personal authentication. Methods: In this article, a novel procedure for online signature verification and recognition has been presented based on DualTree Complex Wavelet Packet Transform (DTCWPT). Results: In the presented method, threelevel decomposition of DTCWPT has been computed for three time signals of dynamic information including horizontal and vertical positions in addition to the pressure signal. Then, in order to make feature vector corresponding to each signature, log energy entropy measures have been computed for each subband of DTCWPT decomposition. Finally, to classify the query signature, three classifiers including knearest neighbor, support vector machine, and Kolmogorov–Smirnov test have been examined. Experiments have been conducted using three benchmark datasets: SVC2004, MCYT100, as two Latin online signature datasets, and NDSD as a Persian signature dataset. Conclusion: Obtained favorable experimental results, in comparison with literature, confirm the effectiveness of the presented method in both online signature verification and recognition objects.
Keywords: Dualtree complex wavelet packet transform, Kolmogorov–Smirnov test, log energy entropy measure, online handwritten signature verification, signature recognition
How to cite this article: Foroozandeh A, Hemmat AA, Rabbani H. Online handwritten signature verification and recognition based on dualtree complex wavelet packet transform. J Med Signals Sens 2020;10:14557 
How to cite this URL: Foroozandeh A, Hemmat AA, Rabbani H. Online handwritten signature verification and recognition based on dualtree complex wavelet packet transform. J Med Signals Sens [serial online] 2020 [cited 2021 Jul 23];10:14557. Available from: https://www.jmssjournal.net/text.asp?2020/10/3/145/288895 
Introduction   
Conventionally, authentication of a person was conducted in two types, including knowledgebased or tokenbased methods. In case of knowledgebased authentication system, the authentication is conducted based on something the user knows; the answer of the user to the secrete question(s) such as personal identification number or password.^{[1]} However, tokenbased authentication systems work based on something the user has, such as a driver's license or identity document card.^{[1]} However, traditionally authentication systems suffer from several treats including forgotten or stolen. These issues are addressed using biometrics. Authentication of a person using physical or behavioral characteristics is known as biometric. In the case of physical biometrics, authentication has been conducted using direct measurements of a part of human body, such as fingerprint,^{[2]} face,^{[3]} and iris.^{[4]} On the other hand, behavioral biometrics use the information of an action performed by the user such as voice,^{[5]} gait,^{[6]} and signature.^{[7]} Biometricbased authentication systems use many different aspects of human physiology, chemistry, or behavior. However, selection of a suitable biometric depends on several factors including:
 Robustness; does not have substantial changes over time
 Distinctively; has a great variation over different subjects
 Availability; all intended people have this characteristic
 Accessibility; easy to collect
 Acceptability; intended people agree to be taken from them.^{[8]}
Handwritten signature is one of the most valuable biometric traits, which has mostly been used for verification purposes in everyday life.^{[9]} Financial and administrative institutions use handwritten signatures as legal means of verifying an individual's identity, all over the world.^{[9]} In addition, handwritten signature has an important advantage of noninvasive and nonthreatening process by the majority of the users.^{[10]} It should be noted that signatures written by different people are naturally different which is known as interclass variability. Furthermore, due to the impact of physical and emotional conditions of the person on its signatures, handwritten signatures of one person are not the same and this variability is known as intraclass variability.^{[10]} Notably, in addition to high intraclass variability, some other disadvantages are included in the case of handwritten signatures: forgeries, higher error rates than other biometrics, and large temporal variation.
A biometric system can be operated in two modes, namely recognition (identification) or verification. The aim of a verification system is the veracity of the person's claimed identity. However, an identification system tries to recognize the identity of the user.^{[8]} A signature verification system deals with three kinds of forgeries: random, simple, and skilled forgeries.^{[11]} In the case of random forgery, the forger without any information about the author's name and his/her signature reproduces a random signature. If the forger knows the author's name but does not access to any signature sample, the reproduced signature is known as a simple forgery. In the case of skilled forgery, a forger has signature samples and tries to reproduce them. Therefore, skilled forgeries are much more similar to the genuine signatures than random and simple forgeries. The focus of this paper, in the case of verification, is on the verification of skilled forgery samples.
It should be noted that training in an automatic signature verification system may be writerindependent (WI) or writerdependent (WD).^{[12]} In the first case, WI, training is conducted based on a large population of signature samples related to all persons in the dataset, whereas in the case of WD, training is done based on the signature samples of each person, separately.^{[12]} Although WD approach achieves good results, for each user added to the system, a classifier must be conducted again which increases the complexity and cost of the system.^{[13]} To reduce the complexity, WI approach attracts more researchers in recent years.^{[13],[14]} In this article, we consider WI approach and a novel procedure for online handwritten signature verification and recognition is presented.
Several reviews of the stateoftheart on signature identification and signature verification have been recently proposed.^{[15],[16],[17]} Input data of the signaturebased authentication systems are in two types: offline (static) and online (dynamic). Offline handwritten signaturebased authentication systems receive only the images of signatures gathered by a camera or a scanner. However, an online signaturebased authentication system receives dynamic (temporal) information of the signing process in addition to the signature images which are gathered by a digitizing tablet or a pensensitive computer. Dynamic information includes time signals of horizontal position, x(t), vertical position, y(t), velocity, v(t), pressure, p(t), pen azimuth, z (t), and altitude, l (t).^{[10]}
There are two main categories for feature extraction, i.e., information extracted from online signatures: global and functional. The aim of a global feature extraction method is to derive feature vectors of the same length and compare them.^{[18]} Global feature extraction method divides into two categories: In the first category, features are extracted from the totally of the signature such as average pressure, average velocity, pen tip, total signing duration, and signature height,^{[18],[19]} while second category is dedicated to the features extracted by applying a transformation on the signature such as discrete wavelet transform (DWT) and discrete cosine transform.^{[20],[21]} On the other hand, functional features are dedicated to the features extracted by considering time sequences describing the signing process and calculating the distances between them.^{[18]} Functional features are in two types; in the first type, a set of reference samples related to each subject has been saved as reference set. Then, classifying is done with the comparison of the input signature with the reference set using some methods such as Dynamic Time Warping (DTW). In the second type, a model is trained using the saved signature, and then, the test signature is classified using the trained model.^{[18]}
An interesting field of research in engineering, mathematics, and bioinformatics is multiscale modeling. Multiscale modeling deals with problems with multiscale nature, i.e., having important features at multiple scales of time and/or space.^{[22]} Notably, signals represented in time domain have important properties in the frequency domain.^{[23]} Analysis of a time signal for its frequency content is effectively done using Fourier transform, wavelets, and other xlet transforms such as curvelet and contourlet.^{[23],[24]} Here, a novel signature verification/recognition method using DTCWPT decomposition to obtain the frequency content of dynamic information of online signatures, has been presented.
The rest of this paper is organized as follows: a brief review on DTCWPT with a literature review are presented in Subjects and Methods section. Details of the presented online signature verification method with the utilized datasets are explained in the Proposed online signature verification method section. Experimental results of signature verification and the comparison with literature are reported in the Experimental protocol and performance evaluation section. The presented online signature recognition method with the experimental results and comparison with literature are discussed in the Proposed online signature recognition method section. Finally, we conclude our work and outline some future work directions in the Conclusion and Future Works section.
Subjects and Methods   
This section is dedicated to a brief review on basic concept used in the presented method, i.e., DTCWPT, with a literature review on the related signature verification methods.
A brief review on dualtree complex wavelet packet transform
Fourier transform is one of the most popular tools used in the field of multiscale modeling. In the Fourier transform, infinitely oscillating sinusoidal basis functions are used to represent the input signal. Some of the limitations of Fourier transform have been modified using wavelet transform.^{[20]} Indeed, oscillating sinusoidal basis functions used in the Fourier transform are replaced with wavelets as basic functions which are locally oscillating in the DWT.^{[20]} Wavelets are stretched and shifted versions of a realvalued bandpass wavelet Ψ(t). To form orthonormal basis for the space of onedimensional realvalued continuoustime signals, wavelets are combined with shifted versions of a realvalued lowpass scaling function φ(t). Therefore, any finite energy signal x(t) can be decomposed in terms of wavelets and scaling function as follows:
Where φ(.) and Ψ(.) are computed using lowpass and highpass filters, h_{0}(n) and h_{1}(n), respectively, as follows:
Further, the scaling coefficients, c(n), and wavelet coefficients, d(j, n), are computed using the inner product, as follows:
In DWT, only low frequency band is decomposed at each level of decomposition. The procedure of a twolevel decomposition of DWT has been shown in [Figure 1]a. DWT has some advantages including good compression, perfect reconstruction, no redundancy, and very low computation.^{[25]} However, DWT suffers from four deficiencies, including oscillations, shift variance, aliasing, and lack of directionality.^{[26]}  Figure 1: The structure of a twolevel decomposition of a discrete wavelet transform (a) and dualtree complex wavelet transform (b) and the structure of the first wavelet packet of a twolevel decomposition of a dualtree complex wavelet packet transform (c)
Click here to view 
An octaveband analysis of the frequency domain is provided by DWT which might not be optimal for a given signal. Finding an optimal representation is possible with discrete wavelet packet transform (DWPT).^{[27],[28]} In DWT, only low frequency band is decomposed at each level, but in DWPT, decomposition is conducted for both low and high frequency bands. It should be noted that DWPT is also shiftvarying and suffers from lack of directionality in twodimensions (2D) and higher dimensions, like DWT. These deficiencies are approximately addressed by dualtree complex wavelet transform (DTCWT).^{[28]}
The structure of DTCWT is composed of two DWTs; coefficients of the first and second DWTs are combined as real and imaginary parts of DTCWT. Suppose that the first and second wavelets are denoted by Ψ(t) and Ψ'(t), respectively. These wavelets are defined similar to Eq. 2 and Eq. 3 using {h_{0}(n), h_{1}(n)} and {h_{0}'(n), h_{1}'(n)}, for Ψ(t) and Ψ'(t), respectively. [Figure 1]b shows the structure of twolevel decomposition of DTCWT. It should be noted that Hilbert transform of a function U(t), is computed as follows:
To improve the shiftability of DWT, in DTCWT, the second wavelet, Ψ'(t), is the Hilbert transform of the first wavelet,Ψ(t), i.e. Ψ' = H{Ψ(t)}. Advantages of DTCWT include low computation, limited redundancy, perfect reconstruction, good directional selectivity, good shift invariance, and analyticity in 1D.^{[25]}
As the DWPT extending the DWT, DTCWPT extends the DTCWT.^{[28]} DTCWPT is obtained by iterating two perfect reconstruction filter banks on the lowpass and highpass outputs.^{[28]} The design of these filters is such that the response of each branch of the second wavelet packet filter bank is the discrete Hilbert transform of the corresponding branch of the first wavelet packet filter bank. Therefore, each subband of the DTCWPT will be approximately analytic.^{[28]} This is true when the filters used in the second wavelet packet of DTCWPT are the same with the filters used in the first wavelet packet of it. Notably, similar to DTCWT, approximately shiftinvariance and good directional selectivity are provided with DTCWPT. [Figure 1]c shows the structure of twolevel decomposition of DTCWPT. As shown in [Figure 1]c, for klevel decomposition of DWT, DTCWT, and DTCWPT of ID signal, the number of output subbands is k + 1, 2(k + 1), and 2^{k + 1}, respectively.
Literature review
Traditional wavelets and extensions of wavelets, such as DWPT and DTCWT, have been widely used in the structure of the presented signaturebased authentication systems, until today. In the following, some of the presented online signature verification methods based on wavelet or similar transforms have been briefly reviewed. An online signature verification system based on DWT features and neural network classification has been presented in the study by Maged and Fahmy.^{[21]} Threelevel decomposition of DWPT has been implemented on time signals, including horizontal and vertical position, pressure, pen azimuth, and pen altitude, in the study by Wang et al.^{[29]} Then, different combinations of these decompositions have been considered as extracted features. Finally, decision about genuine or forgery of the query signature has been made using the Euclidean distance. Continuous wavelet transform has been used to obtain the frequency information of the speed signals in.^{[30]} The grayscale spectrograms created by wavelet transforms have been used to train support vector machine (SVM) network. In the study by Chang et al.,^{[31]} five time signals including horizontal and vertical position, pressure, pen azimuth, and altitude have been considered for each signature. Then, different wavelets such as Haar, Daubechies, Symlet, and Coiflet have been used to decompose the time signal up to fivelevel decomposition. Final decision about the query signature has been conducted using a threshold on the computed distance measures. An improved waveletbased online signature verification scheme has been presented in the study by Nilchiyan and Yusof.^{[32]} In this work, five time signals including horizontal and vertical position, pressure, pen azimuth, and altitude have been discussed, and threelevel decomposition of them using Haar and Daubechies wavelets have been computed for feature extraction. Classification has been done using the multiperceptron neural network with one intermediary layer.
It should be noted that, in comparison with signature verification, there are fewer works conducted for online signature recognition, until today. To the best of our knowledge, there is no online signature recognition work based on wavelet or other similar transforms. However, there are few other procedures for online signature recognition, which have been briefly reviewed in the following. An online signature slant identification was presented in the study by Shamsuddin and Mohamed.^{[33]} Horizontal and vertical positions of the signing process have been considered and filtered them. Then, the angle and degree of the signature have been computed and the signature has been classified into its slant category using a slant algorithm. This work presented an accuracy of 80% on a private dataset. In the study by Mohamed et al.,^{[34]} a baseline extraction algorithm has been used for online signature recognition based on vector rules. Direction, slant, baseline, pressure, speed, and numbers of pen ups and downs have been used as the main features. An algorithm to extract baseline from signature has been used and obtained the accuracy of 90% on their created dataset. An online signature recognition based on some global features such as standard deviation and maximum of the velocity and the acceleration in both x and y directions has been presented in the study by AlMayyan et al.^{[35]} A rough set classifier has been used and the accuracy of 100% for recognition rate has been obtained on its own created dataset composed of 2160 signatures from 108 subjects. Concepts of graph theory have been used for online signature identification in the study byFotak et al.^{[36]} A fast classification using graph norm and comparison between each signature graph concepts value with the saved values in the dataset has been conducted to obtain an identification accuracy of 94.25% on their own dataset.
However, to the best of our knowledge, DTCWPT has not been used in the structure of online signature recognition and verification systems, until today. In this paper, a writerindependent online handwritten signature recognition and verification system is presented based on DTCWPT decomposition of time signals containing dynamic information of each signature.
Proposed online signature verification method
In this section, the novel presented method for online handwritten signature verification based on DTCWPT has been described in detail. The block diagram of the presented signature verification method is shown in [Figure 2].  Figure 2: Block diagram of the presented online signature verification method
Click here to view 
Online handwritten signature datasets
The performance of the presented online signature verification method has been evaluated using three benchmark available datasets, including SVC2004^{[37]} and Ministerio De Ciencia Y Tecnologia (MCYT)100,^{[38]} as two benchmark online handwritten Latin signature datasets, and Noshirvani Dynamic Signature Dataset (NDSD),^{[39]} as an online handwritten Persian signature dataset. The statistics of these datasets are displayed in [Table 1].
SVC2004 (https://www.cse.ust.hk/svc2004/) is an online handwritten Latin signature dataset established for the First International Signature Verification Competition (SVC2004).^{[37]} WACOM INTUOS tablet was used for saving the dynamic information of the signing process. SVC2004 dataset provided two different signature databases, namely Task 1 and Task 2. The main SVC2004 dataset includes 100 contributors: 40 persons included in Task 1 and other 60 persons included in Task 2 signature dataset. Task 2, which has been considered in this study, contains 40 signers with 20 genuine signatures and 20 skilled forgeries per signer. Genuine signatures were signed in 2 weeks and skilled forgeries were signed with four other contributors for each contributor's signature. Dynamic information saved in Task 2 used in the presented method is shown in [Table 1].
MCYT (http://atvs.ii.uam.es/atvs/mcyt100s.html) dataset composed of a fingerprint and an online Latin signature dataset.^{[38]} Here, the public subset of MCYT contains signatures of 100 signers (MCYT100) has been used. Dynamic information shown in [Table 1] was saved using WACOM pen tablet, model INTUOS A6 USB with a capture area of size 127 × 97.^{[38]} There are 25 genuine signatures and 25 skilled forgeries for each individual enrolled in the dataset. Forgeries were produced by five subsequent target users trying to copy the statistic images of the signature in 10 times.
NDSD (https://sites.google.com/site/meyahyatabar/ND SD % 20 Data.rar?attredirects = 0 and d = 1) is an online handwritten Persian signature dataset. WACOM INTUOS4 digitizing tablet was used to save dynamic information of signing process.^{[39]} Signers signed on a plate of size 96 × 129 and the information was saved every 10 ms. These dynamic informations are shown in [Table 1]. Signers were in the range of 18–40 years old and signed 65 signatures in two different times with more than 3 days' interval. Further, 15 professional forgers forged 40 samples from 65 genuine signatures for each person.^{[39]} As an example, one sample of online signature image from MCYT100 signature dataset with the corresponding signals of horizontal position (xcoordinate), vertical position (ycoordinate), and pressure is shown in [Figure 3].  Figure 3: One Signature sample from online signature datasets Ministerio De Ciencia Y Tecnologia100, with the corresponding dynamic information including xcoordinate, ycoordinate, and pressure
Click here to view 
Preprocessing
Among dynamic information describing the signing process available in the used datasets [Table 1], only three time signals including xcoordinate (x(t)), ycoordinate (y(t)), and pressure (p(t)) have been considered in the presented online signature verification method. Due to the intraclass variability, time signals corresponding to each signer have different length and size. Hence, some preprocessing tasks including length normalization, rotation normalization, and size normalization have been conducted on the mentioned time signals. Length normalization has been conducted to equalize the number of points of each signal. This is done using resampling of the signal. Further, in the process of gathering signatures, handwritten signatures are recorded in different angles, and so, rotation normalization is necessary to reduce intraclass variability. Rotation normalization has been conducted similar to.^{[40]} In addition, since different dynamic ranges and to minimize this range and set the value changes in a specified interval, Minmax normalization has been conducted to set the values in (0,100) using Eq. 7, and then, they become standard using Eq. 8, as follows:^{[35]}
Where f(t) is the values of a time signal and f_{Min} and f_{Max} are the minimum and maximum of the signal in hand, respectively. Notably, μ and σ are the average and standard deviation of the signal.
Proposed online signature verification method
After conducting the preprocessing tasks on the time signals including xcoordinate, ycoordinate, and pressure, feature extraction step has been conducted. Due to the intraclass and interclass variabilities, an effective signature verification system should be considered features with the ability to minimize intraclass variability and maximize interclass variability. The best way to do this is using the multiscale modeling.^{[21],[29]} In this article, a global feature extraction approach using DTCWPT as multiscale modeling tool has been considered. To extract features from time signals, threelevel decomposition of DTCWPT has been performed on each time signal. The output subbands include the local characteristics of the original signal in threelevel depth of decomposition. However, the number of these coefficients is so huge to directly use as the classifier input.^{[41]} Therefore, some features should be derived from these coefficients to enhance the performance of the classifier.
To extract features, entropy concept has been used. Entropy is a common concept in many fields of research, mainly in signal processing,^{[41]} and it is one of the widely used tools to extract features. Entropy measures the uncertainty of the information included in the signals.^{[41]} There are several types of entropy including Shannon, Renyi, Tsallis, and log energy.^{[41]} Shannon entropy of a time signal as a wellknown entropy is a measure of its average uncertainty and computed as follows:
Where f_{i} is the value of the time signal f(t) at time t_{i}. Shannon entropy has been used in the structure of the signature verification system, until today.^{[42],[43]} In this article, the log energy entropy (LEE) has been used as follows:
To extract features, LEE for all subbands obtained from threelevel decomposition of DTCWPT has been computed. DTCWPT is composed of two wavelet packets and each of them yields to 16 (=2^{3+1}) subbands. Hence, totally, 32 (=16 × 2) subbands have obtained with threelevel decomposition of DTCWPT. LEE of all of these subbands has been computed and concatenated to each other to make a feature vector corresponding to the original signature.
To classify the signatures in the feature space, three approaches have been tested for classification: knearest neighbor (kNN), SVM, and KolmogorovSmirnov (KS) test as a statistical classification approach. kNN is a nonparametric classification technique, i.e., it does not make any assumptions on the underlying data distribution.^{[44]} kNN works based on feature similarity and its output is a class membership, a discrete value to predict a class by a majority vote of its neighbors, and being assigned to the class most common among its kNNs (https://blog.usejournal.com/aquickintroductiontoknearestneighborsalgorithm62214cea29c7).^{[44]}
SVM classification as a supervised learning algorithm is one of the most popular classifiers used in machine learning, pattern recognition, and signal processing.^{[45]} The idea behind the SVM classification is finding a hyperplane with best separation between the data in the feature space.^{[46]} In the case of nonlinear decision function, a nonlinear transformation is used to map the data from the input feature space into a highdimensional feature space. According to the Cover's theorem and for nonlinear transformation and highdimensional space, the patterns are linearly separable with high probability in the new feature space.^{[45]} This nonlinear transformation is performed using kernel functions. There are different types of kernel including linear, polynomial, and radial basis function (RBF). RBF is one of the most popular kernel functions whose value depends on the distance from the origin or from some point. RBF kernel for two feature vectors X and X' is defined as follows:
Where ‖ ‖^{2} is the squared Euclidean distance and σ is a free parameter. Equivalently, with , we have:
KS test is a nonparametric hypothesis test which is widely used for comparing two given distance distributions of a single independent variable.^{[47]} The KS static for comparing two distributions P_{0} and P is computed as follows:^{[47]}
Where F_{m}(.) and G_{n} (x) are empirical cumulative distribution functions (c.d.f.) of two given distance distributions P_{0} and P, respectively. The P value for this test of hypothesis is computed as follows:
Where H(.) is the c.d.f. of KS distribution with the following definition:^{[47]}
KS test for two samples tests the null hypothesis , i.e., two samples come from the same distribution and accept the null hypothesis, if the computed P value is greater than a significance level (α), else the null hypothesis is rejected. It should be noted that in testing hypothesis, two types of error can occur: type I and type II.^{[47]} Type I error occurs if the null hypothesis is rejected when it is true, and type II error occurs if the null hypothesis is accepted when it is false. The probability of committing a type I error in a decision rule is called the significance level (α). Significance level is usually considered as 5%, or 1%, and the null hypothesis is accepted with confidence level 95% (= 1 − α, for α = 5%).^{[47]}
Experimental protocol and performance evaluation
In this section, details of the conducted experiments for the presented online signature verification in addition to the obtained experimental results have been described. At the end of this section, the presented method has been compared with literature.
Experimental protocol for the proposed online signature verification method
Three time signals xcoordinate, ycoordinate, and pressure have been considered as dynamic information corresponding to each signature in the dataset. Preprocessing tasks including length normalization, rotation normalization, and size normalization have been conducted on the mentioned time signals. Feature extraction has been conducted based on DTCWPT on the preprocessed data. DTCWPT has two separate wavelet packets. To get features from the different scales of the signals, threelevel decomposition of DTCWPT has been performed on the signals. Therefore, 16 subbands have been obtained for each signal at each wavelet packet. Notably, the length of these subbands is dependent to the length of the input signal which is different between datasets and also their users. The number of these coefficients is so huge to directly use as the classifier input, and therefore, LEE of each DTCWPT subband has been computed as features. These features have been computed for 32 all subbands from both first and second wavelet packets of the threelevel decomposition of a DTCWPT for every signature in the dataset and concatenated in a vector as a feature vector of the signature. Therefore, for each time signal, 32 entropy measures have been computed as features per time signal. Totally, with considering three mentioned time signals, the final feature vector of length 96 (= 3 × 32) has been considered for each signature in the dataset. These feature vectors have been computed for every genuine signatures and skilled forgery samples in the dataset and then fed into the classification step.
Three classification techniques including kNN, SVM, and KS test have been considered to make decision for acceptance/rejection of the test signature. The parameters of these classification techniques have been empirically set as follows; kNN with k = 3 neighbors and SVM with RBF kernel and regularization parameter C = 1000. Regularization parameter trades off the correct classification of training examples against maximization of the decision function's margin (https://www.mathworks.com/help/stats/fitcsvm.html). These values of the parameters work well for the problem; however, we noted that they could be optimized which is not explored in this work.
Here, an online handwritten signature verification with the WI approach has been presented. It should be noted that there are no skilled forgeries for each user enrolled to the system in real situations. Hence, to applicability of the presented system, training of the classifier only conducted with genuine signatures available in the dataset. To train the classifier and to avoid overfitting, feature vectors of genuine signatures have been divided into three sets: training, validation, and testing with 60%, 20%, and 20% of data, respectively. Then, the feature vectors of all of the forgery samples have been added into testing set and the final results have been reported using threefold cross validation.
In the case of KS test, making decision for acceptance/rejection of the query signature is formulated as a testing hypothesis, i.e., decision between two existed status: the query signature belongs to the person having claimed his/her identity (null hypothesis) or it is a forged sample (alternative hypothesis) (https://machinelearningmastery.com/statisticalhypothesistests/). In other words, KS test is a decision between the following hypotheses:
Where P_{0} is considered as the distribution of distances between training samples of true person, and P is considered as the distribution of distances between the query signature with all true saved samples of the person having claimed. Two distributions P_{0} and P include pairwise distances between feature vectors of the signatures. These distances have been computed using Euclidean distance, in this article. Naturally, two distance distributions P_{0} and P are slightly different. However, in signature verification process, the goal is answering to the question that whether existed difference is significant for failing the null hypothesis (rejection the query signature) or not. To do this, KS test with α = 5% has been considered in our experiments. The signature in question has been claimed as a genuine signature (accept the null hypothesis) or a skilled forgery sample (reject the null hypothesis) with confidence level of 95% (= 1 − α). Obtained experimental results are provided in the next section.
Performance evaluation of the proposed online signature verification method
The performance of the presented signature verification method has been evaluated using three wellknown criteria: false rejection rate (FRR), false acceptance rate (FAR), and equal error rate (EER). FRR is the fraction of the genuine signatures which have been falsely rejected, FAR is the fraction of skilled forgeries falsely accepted, and EER is the error rate when FRR = FAR. The presented signature verification method has been implemented using a system with Intel Core i77500U for CPU and 12 GB of RAM. Here, the MATLAB implementation of DTCWPT has been used with threelevel decomposition (https://ilkerbayram.github.io/dtcwpt/),^{[28]} and the implementation of the presented method has been conducted using the programming language MATLAB R2017a (Natick, Massachusetts, USA).
Online signature verification results and comparison with literature
Several experiments have been conducted using three datasets of two scripts, including SVC2004 and MCYT100, as two publicly available benchmark Latin datasets, and NDSD as a benchmark Persian dataset. Obtained experimental results are shown in [Table 2] for three classification techniques: kNN, SVM, and KS test. As shown in [Table 2], among three classification techniques, the best results in all three datasets have been obtained using KS test for decisionmaking about genuine/forgery of the query signature. After KS test, SVM with RBF kernel and kNN classification obtained the best results, respectively.  Table 2: Obtained signature verification results on the three benchmark datasets (%)
Click here to view 
To better intuition about how to separate the genuine signatures from forgery samples in the space of their feature vectors, tdistributed stochastic neighbor embedding (tSNE) (https://lvdmaaten.github.io/tsne/) algorithm has been used. tSNE algorithm is a successful method which has been presented for dimensionality reduction introduced by van der Maaten and Hinton in 2008.^{[48]} tSNE technique is used for visualizing highdimensional data into a lowdimensional space of two or three. Here, tSNE algorithm has been used to visualize the feature vectors from feature space of 96dimensional into 2D space. [Figure 4] shows 2D projections of the feature vectors corresponding to the 65 genuine signatures and 40 skilled forgeries related to one user in NDSD dataset. Further, [Figure 5] shows 65 genuine signatures and 40 skilled forgeries related to four different users in NDSD dataset. As shown in these [Figure 4] and [Figure 5], the extracted features can make good separation between genuine signatures and skilled forgeries, which verifies the favorable signature verification results.  Figure 4: Separation of 65 genuine signatures from 40 forgery samples corresponding to one user from Noshirvani Dynamic Signature Dataset in the feature space, using tdistributed stochastic neighbor embedding algorithm. Genuine signatures have been shown using blue circle and the corresponding forgery samples have been shown with red “+” marker
Click here to view 
 Figure 5: Separation of 65 genuine signatures from 40 forgery samples corresponding to four users from NDSD dataset in the feature space, using tSNE algorithm. An obvious separation between four users is seen, in this figure. The pairwise markers (red 'o' and green '+'), (yellow '.' and black square), (violet '♦' and dark blue square), and (yellow '▾' and blue '.') have been used to show genuine signatures and forgery samples related to each user
Click here to view 
To compare the obtained experimental results with literature, several prominent online handwritten signature verification methods have been selected and compared using [Table 3]. [Table 3] compares the presented signature verification method with some prominent methods implemented on three datasets: SVC2004, MCYT100, and NDSD. As shown in [Table 3], in the case of MCYTY100 and NDSD datasets, the presented signature verification method outperformed the stateoftheart. In the case of SVC2004, the stateoftheart is dedicated to the presented method by Fayyaz et al.^{[49]} It should be noted that Fayyaz et al.^{[49]} used taught learning to learn feature from 17500 signature images of ATVS dataset,^{[50],[51]} and verification process has been conducted on SVC2004 to get 0.83% for EER. In comparison with this method, our presented method has lower computational cost which using handcrafted features, instead of learning features.^{[49]} However, [Table 3] shows that in comparison with other prominent works, our method also obtained promising results in the case of SVC2004.  Table 3: Comparison of the proposed online signature verification method with literature on three datasets: SVC2004, Ministerio De Ciencia Y Tecnologia100, and Noshirvani Dynamic Signature Dataset
Click here to view 
Proposed online signature recognition method
In this section, the proposed procedure for online signature recognition with the corresponding experiments, obtained results, and comparison with literature has been presented.
Proposed online signature recognition method
In this section, an online handwritten signature recognition has been presented. Similar to the presented signature verification method, three time signals xcoordinate, ycoordinate, and pressure have been considered and the preprocessing tasks have been conducted on them for each signature in dataset. Further, to extract features, a threelevel decomposition of DTCWPT has been performed on the mentioned three time signals. Then, entropy measures using LEE have been computed for each of the obtained subbands of DTCWPT decomposition. These entropies have been concatenated to each other to make the feature vector related to each signature in dataset. Finally, to recognize the class of the query signature in the presented signature recognition method, two classifiers including kNN and SVM have been considered.
Online signature recognition results and comparison with literature
Three benchmark online signature datasets, including SVC2004 and MCYT100, as two Latin datasets, and NDSD, as a Persian dataset, have been considered for experiments. Notably, in the case of signature recognition, only genuine signatures from each dataset have been used. The presented signature recognition method has been evaluated using true recognition rate, i.e., the percentage of truly classified signatures on the datasets. To classify the query signature, two classifiers with the following parameters have been used: kNN with k = 3 and SVM with RBF kernel and the regularization parameter C = 1000. These parameters have been empirically set for our experiments. Notably, in the case of SVM classifier, in addition to RBF kernel, polynomial kernel and linear kernel have been used to evaluate the presented signature recognition method. However, the best results have been obtained by RBF kernel of SVM classifier. To avoid overfitting, final signature recognition results have been reported using threefold cross validation.
To the best of our knowledge, KS test has not been used for recognition purpose, until today. This also has not been used here. Authors believe that this is because of the high computations and inaccurate decisions that may occurred with KS test in the recognition decisions. This is explained here. To recognize the class of the query signature, the comparison should be done with all classes of people registered in the system. Suppose that there exist M classes of writers. Therefore, computed pairwise distances between signatures in each class lead to make M distributions of distances. In addition, computed distances between the query signature and the signatures in each class make M other distributions. KS static should be computed between distribution of one class as P_{0}, and distribution of the query signature and that class, as P_{1}. The computed KS static is compared to the considered P value. Then, between all of the acceptable KS statics, the class of the query signature has been selected as the class with the maximum KS static. Therefore, as explained, the computations are increased and make a final decision can be confusing when some equal KS statics have been computed.
The presented signature recognition method has been implemented using a system with Intel Core i77500U for CPU and 12 GB of RAM. Here, the MATLAB implementation of DTCWPT has been used with threelevel decomposition,^{[28]} and the implementation of the presented method has been conducted using the programming language MATLAB R2017a.
[Table 4] shows the obtained signature recognition results. As shown in [Table 4], the best recognition rate has been obtained using SVM with RBF kernel on three considered datasets.  Table 4: Obtained signature recognition results on the three benchmark datasets in terms of true recognition rate (%)
Click here to view 
To visualize signature recognition results, tSNE algorithm has been used to better intuition about how to separation of signatures from different classes in the space of the feature vectors. Here, tSNE algorithm has been used to visualize the feature vectors from feature space of 96 dimensions into 2D space. As an example, eight users from NDSD dataset have been considered and 2D projections of the feature vectors corresponding to 30 signatures of each user have been shown using tSNE algorithm, in [Figure 6]. As shown in [Figure 6], there is a geed separation between genuine signatures from eight different classes in their feature vectors, which leads to obtain favorable online signature recognition results shown in [Table 4].  Figure 6: Separation of 240 signatures of 8 users (30 signatures per user), from Noshirvani Dynamic Signature Dataset in the feature space, using tdistributed stochastic neighbor embedding algorithm. Signatures from each class have been shown with different markers and discriminative colors
Click here to view 
To compare the signature recognition results with literature, some of the prominent presented signature recognition methods have been considered. It should be noted that, in comparison with literature on signature verification methods, fewer methods have been presented for online signature recognition. To the best of our knowledge, all of these methods have been evaluated on a private dataset and the comparison is not fair. However, their obtained true recognition rates and our obtained recognition rate are shown in [Table 5]. As shown in [Table 5], the previous works presented good results and the method in the study by AlMayyan et al.^{[35]} obtained the accurate recognition rate on its own created dataset composed of 2160 signature from 108 users. However, obtained signature recognition results on three benchmark dataset verify the suitably and effectiveness of the presented signature recognition method.  Table 5: Comparison of the proposed online signature recognition method with literature in terms of true recognition rate (%)
Click here to view 
Conclusion and Future Works   
In this paper, a novel approach for online handwritten signature verification and recognition has been presented. Dynamic information including three time signals xcoordinate, ycoordinate, and pressure have been considered to extract features. DTCWPT has been used as a multiscale transform with threelevel decomposition of the time signals. Using this decomposition, 32 subbands have been obtained for both of two wavelet packets of DTCWPT. Then, instead of the huge number of coefficients of DTCWPT subbands, entropy measures of each subband have been computed as feature corresponding to that subband. In the presented method, LEE has been used and the computed features of each subband have been concatenated to make the feature vector related to each signature in dataset. Three classifiers KNN with k = 3, SVM with RBF kernel, and KS test with confidence level of 95% have been used to final decision about the query signature to be a genuine signature or considered as a forgery sample. The performance of the method has been evaluated on three publicly available benchmark datasets: SVC2004 and MCYT100 as two Latin datasets and NDSD as a Persian dataset. Experimental results show that the best classification has been conducted using KS test on three mentioned datasets. Comparison with literature verifies that the presented method using DTCWPT and LEE outperformed the stateoftheart on MCYT100 and NDSD datasets and also obtained really promising results on SVC2004 dataset.
Similar to the presented signature verification method, an online signature recognition method has been presented. After conducting the preprocessing step on three time signals: xcoordinate, ycoordinate, and pressure, their corresponding feature vectors have been computed using threelevel decomposition of DTCWPT and LEE measures. The classification for recognition of the user's class has been conducted using kNN and SVM classifiers and the final reported results have been obtained by threefold cross validation. Obtained results show favorable accuracy around 99% on three benchmark datasets which verifies the effectiveness of the presented method.
A possible future work for improving the performance of the presented online signature verification and recognition systems is using other multiscale transforms, other types of entropy measures, and other statistical test in the structure of the systems. In addition, the performance of DTCWPT will be evaluated in the case of offline handwritten signature verification and recognition, by us.>
Acknowledgment
We would like to thank the reviewers for their valuable suggestions and comments during the revision process.
Financial support and sponsorship
None.
Conflicts of interest
There are no conflicts of interest.
Biographies   
Atefeh Foroozandeh, received her B.Sc. and M.Sc. degrees in Applied Mathematics from Ferdowsi University of Mashhad, Iran, in 2007 and 2010, respectively. She is now a PhD student of Applied Mathematics, Wavelet Analysis in Graduate University of Advanced Technology, Kerman, Iran. Her current research interests include handwritten signature recognition and verification (online and offline), deep learning, and the application of Xlets, sparse transforms and fractal theory in the field of signal and image processing.
Email: [email protected]
Ataollah Askari Hemmatreceived received his B.Sc. degree in Mathematics and his M.Sc. degree in Analysis from Shahid Bahonar University of Kerman, Kerman , Iran, in 1987 and 1990, respectively. He received his Ph.D. degree in Analysis (Wavelets) from Shiraz University, Shiraz, Iran in 2000. He is now a full professor in Department of Applied Mathematics, Faculty of Mathematics and Computer, Shahid Bahonar University of Kerman. His field of specialization include Functional Analysis, Fourier Analysis, Wavelet, Frame Theory, and Harmonic Analysis. His research interests are Analysis, wavelets, and application of wavelets in solving differential equations.
Email: [email protected]
Hossein Rabbani received the B.Sc. degree in Electrical Engineering (Communications) from Isfahan University of Technology, Isfahan, Iran, in 2000 with the highest honors, and the M.Sc. and Ph.D. degrees in Bioelectrical Engineering in 2002 and 2008, respectively, from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran. In 2007 he was with the Department of Electrical and Computer Engineering, Queen's University, Kingston, ON, Canada, as a Visiting Researcher, in 2011 with the University of Iowa, IA, United States, as a Postdoctoral Research Scholar, and in 20132014 with Duke University Eye Center as a Postdoctoral Fellow. He is now a Professor in Biomedical Engineering Department and Medical Image & Signal Processing Research Center, Isfahan University of Medical Sciences, Isfahan, Iran. Dr Rabbani is a Senior Member of IEEE, IEEE Signal Processing Society and IEEE Engineering in Medicine and Biology Society. His main research interests are medical image analysis and modeling, statistical (multidimensional) signal processing, sparse transforms, and image restoration, which more than 110 papers and book chapters have been published by him as an author or coauthor in these areas.
Email: [email protected]
References   
1.  Clarke NL, Furnel SM. Advanced user authentication for mobile devices. J Comput Secur 2007;26:10919. 
2.  Rahman I, Razzaq AH, Ali U. A review on fingerprints recognition system. J Comput Sci Syst Biol 2018;11:2869. 
3.  Chen Z, Huang W, Lv Z. Towards a face recognition method based on uncorrelated discriminant sparse preserving projection. J Multimedia Tools Appl 2015;76:1766983. 
4.  Nguyena k, Fookes C, Jillela R, Sridharana S, Ross A. Long range iris recognition: A survey. J. Pattern Recognit 2017;72:12343. 
5.  Lee Y, Kim S. Study on the situational satisfaction survey of smart phone based on voice recognition technology. J Digit Converg 2017;15:3517. 
6.  Rida I, Almaadeed N, Almaadeed S. Robust gait recognition: A comprehensive survey. IET Biom 2018;8:1428. 
7.  Hafemann LG, Sabourin R, Oliveira LS. Offline Handwritten Signature VerificationLiterature Review. International Conference on Image Processing Theory, Tools and Applications (IPTA); 2017. 
8.  Sharma A. Novel Strategies for Online Signature Verification Using Dynamic Time Warping. Department of Electronics & Electrical Engineering. Thesis. Department of Electronics & Electrical Engineering; July, 2017. Available from: http://gyan.iitg.ernet.in/handle/123456789/986. [Last retrieved on 2019 Aug 31]. 
9.  
10.  Nanni L, Maiorana E, Lumini A, Campisi P. Combining local regional and global matchers for a template protected online signature verification system. Expert Syst Appl 2010;37:367684. 
11.  Arathi M, Govardhan A. An efficient offline signature verification system. Int J Mach Learn Comput 2014;4:5337. 
12.  Srinivasan H, Srihari SN, Beal MJ. Machine learning for signature verification. In: Computer Vision Graphics and Image Processing. Berlin, Heidelberg: Springer; 2006. p. 76175. 
13.  Eskander G, Sabourin R, Granger E. Hybrid writerindependentwriterdependent offline signature verication system. IET Biom 2013;2:16981. 
14.  Guerbai Y, Chibani Y, Hadjadji B. The effective use of the oneclass SVM classifier for handwritten signature verification based on writerindependent parameters. Pattern Recognit 2015;48:10313. 
15.  Sayeed S, Samraj A, Besar R, Hossen J. Online hand signature verification: A review. J Appl Sci 2010;10:163243. 
16.  Zhang Z, Wang K, Wang Y. A survey of online signature verification. In: Biometric Recognition CCBR 2011. Lecture Notes in Computer Science; 2011. p. 1419. 
17.  Yogesh VG. Online signature verification systems: A survey. Int J Emerg Technol 2015;6:637. 
18.  Ahrabian K, Babaali B. On Usage of Autoencoders and Siamese Networks for Online Handwritten Signature Verification. CoRR, arXiv: 1712.02781; 2017. 
19.  FierrezAguilar J, Nanni L, LopezPenalba J, OrtegaGarcia J, Maltoni D. An online signature verification system based on fusion of local and global information. International Conference on Audioand VideoBased Biometric Person Authentication. Berlin, Heidelberg; 2005. p. 523532. 
20.  Yanikoglu B, Kholmatov A. Online signature verification using Fourier descriptors. EURASIP J Adv Signal Process 2009;1:113. 
21.  Maged M, Fahmy M. Online handwritten signature verification system based on DWT features extraction and neural network classification. Ain Shams Eng J 2010;1:5970. 
22.  
23.  Merry RJ. Wavelet Theory and Application: A Literature Study. The Netherlands, Eindhoven: Eindhoven University Technology DCT; 2005. p. 53. 
24.  Starck JL, Murtagh F, Fadili J. Sparse Image and Signal Processing: Wavelets, Curvelets, Morphological Diversity. Cambridge: Cambridge University Press; 2010. 
25.  Kingsbury N. Dualtree Complex Wavelets – Their Key Properties and a Range of ImageProcessing Applications. Signal Processing and Communications Laboratory Department of Engineering, University of Cambridge, UK. Part 1, UDRC Short Course; 21 March, 2013. Available from: https://wwwsigproc.eng.cam.ac.uk/foswiki/pub/Main/NGK/DTCWsApps_UDRC.pdf. [Last retrieved on 2019 Aug 31]. 
26.  Selesnick IW, Baraniuk RG, Kingsbury NG. The dualtree complex wavelet transform. IEEE Signal Process Mag 2005;22:12351. 
27.  Coifman R, Wickerhauser M. Entropybased algorithms for best basis selection. IEEE Trans Inf Theory 1992;38:7138. 
28.  Bayram I, Selesnick IW. On the dualtree complex wavelet packet and Mband transforms. IEEE Trans Signal Process 2008;56:2298310. 
29.  
30.  Alpar O. Online signature verification by continuous wavelet transformation of speed signals. Expert Syst Appl 2018;104:3342. 
31.  Chang H, Dai D, Wang P, Xu Y, Si F, Huang S. Online signature verification using wavelet transform of feature function. J Inform Comput Sci 2012;9:313542. 
32.  Nilchiyan MR, Yusof RB. Improved WaveletBased Online Signature Verification Scheme Considering Pen Scenario Information. First International Conference on. Artificial Intelligence, Modelling and Simulation; 2013. p. 611. 
33.  Shamsuddin MR, Mohamed A. Online signature slant feature identification algorithm. WSEAS Trans Comput Res 2008;3:12130. 
34.  Mohamed A, Yusof R, Rahman SA, Mutalib S. Baseline extraction algorithm for online signature recognition. WSEAS Trans Syst 2009;8:491500. 
35.  AlMayyan W, Own HS, Zedan H. Rough set approach to online signature identification. Digit Signal Process 2011;21:47785. 
36.  Fotak T, Baca M, Koruga P. Handwritten signature identification using basic concepts of graph theory. Trans Signal Process 2011;7:11729. 
37.  Yeung DY, Chang H, Xiong Y, George S, Kashi R, Matsumoto T, et al. SVC2004:First international signature verification competition. In: International Conference on Biometric Authentication. Berlin, Germany: SpringerVerlag; 2004. p. 1622. 
38.  OrtegaGarcia J, FierrezAguilar J, Simon D, Gonzalez J, FaundezZanuy M, Espinosa V, et al. Biometric on the internet MCYT baseline corpus: A bimodal biometric database. IEE Proc Vis Image Signal Process 2003;150:395401. 
39.  Yahyatabar ME, Baleghi Y, Karami MR. Online signature verification: A robust approach for Persian signatures. International Conference on Electrical Engineering. Mashhad, Iran; 2013. p. 16. 
40.  Rashidi S, Fallah A, Towhidkhah F. Authentication Based on Signature Verification Using Position Velocity Acceleration and Jerk Signals. International ISC Conference on Information Security and Cryptology; 2012. p. 2631. 
41.  Li T, Zhou M. ECG classification using wavelet packet entropy and random forests. Entropy 2016;18:285. 
42.  Rosso OA, Ospina R, Frery AC. Classification and verification of handwritten signatures with time causal information theory quantifiers. PLoS One 2016;11:e0166868. 
43.  Choudhary U, Dhurandher SK, Kumar V, Woungang I, Rodrigues JJ. Online Signature Verification Using the Information Set Based Models. 32 ^{nd} International Conference on Advanced Information Networking and Applications. Krakow, Poland; 2018. 
44.  Altman NS. An introduction to kernel and nearestneighbour nonparametric regression. Am Stat 1992;46:17585. 
45.  Byun H, Lee SW. Applications of Support Vector Machines for Pattern Recognition: A Survey. Proceedings 1 ^{st} International Workshop SVM Pattern Recognition with Support Vector Machines; 2002. p. 21336. 
46.  Cortes C, Vapnik V. Supportvector networks. Mach Learn 1995;25:27397. 
47.  Massey FJ. The KolmogorovSmirnov test for goodness of fit. J Am Stat Assoc 1951;46:6878. 
48.  van der Maaten LJ, Hinton GE. Visualizing highdimensional data using tSNE. J Mach Learn Res 2008;9:2579605. 
49.  Fayyaz M, Hajizadeh Saffar M, Sabokrou M, Fathy M. Feature Representation for Online Signature Verification. CoRR abs/1505.08153; 2015. 
50.  Galbally J, Plamondon J, Fierrez J, OrtegaGarcia J. Synthetic online signature generation. Part I: Methodology and algorithms. Pattern Recognit 2012;45:261021. 
51.  Galbally J, Fierrez J, OrtegaGarcia J, Plamondon J. Synthetic online signature generation. Part II: Experimental validation. Pattern Recognit 2012;45:262232. 
52.  Yang L, Jin X, Jiang Q. Online handwritten signature verification based on the most stable feature and partition. Cluster Comput 2019;22:1691701. 
53.  Yahyatabar ME, Ghasemi J. Online signature verification using double stage feature extraction modeled by dynamic feature stability experiment. IET Biom 2017;6:393401. 
54.  Guru DS, Prakash HN. Online signature verification and recognition: An approach based on symbolic representation. IEEE Trans Pattern Anal Mach Intell 2009;31:105973. 
55.  Manjunatha KS, Manjunath S, Guru DS, Somashekara MT. Online signature verification based on writer dependent features and classifiers. Phys Rev Lett 2016;80:12936. 
56.  Diaz M, Fischer A, Ferrer MA, Plamondon R. Dynamic signature verification system based on one real signature. IEEE Trans Cybern 2018;48:22839. 
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6]
[Table 1], [Table 2], [Table 3], [Table 4], [Table 5]
