Activity Recognition Based on Smartphone and Dual-tree Complex Wavelet Transform

Activity Recognition Based on Smartphone and Dual-tree Complex Wavelet

Transform

Chi Wang

Research Institute Electronic Science and Technology University of Electronic Science and Technology of

China

Cheng du, China

wangchiliveevil@https://www.360docs.net/doc/a813643055.html,

Wei Zhang

Research Institute Electronic Science and Technology University of Electronic Science and Technology of

China

Cheng du, China

wzhang@https://www.360docs.net/doc/a813643055.html,

Abstract—Smartphone contains many multiple and powerful sensors, which establishes exciting new opportunities for human-computer interaction and data mining. Those sensors placed inside smartphone are used for phone function enhancement initially. In this work, we show how general machine learning algorithms can use labeled accelerometer data to classify motion activities when users hold a smartphone. First we establish an Android-based data collection application to gain persons’ motion data via accelerometer placed inside smartphone. Then we collect 6 different motion activities from 3 users. Lastly we use normal machine learning algorithm to classify those collected activities. Previous works only use time-domain features for classification. This leads to low accuracy since activity data contains frequency-domain and orientation information. In this paper, we propose a new method for extracting both time-domain and frequency-domain features. We use dual-tree complex wavelet transform (DT-CWT) as feature extraction tool. Then we use general machine learning algorithm tool WEKA for classification. Results show that our method performs better than other researcher’s method which only extracts time-domain feature from accelerometer data in accuracy aspect. Our algorithm acquires accuracy at 86% by using DT-CWT statistical information and orientation feature.

Keywords-component; smartphone; activity recognition; DT-CWT transform

I.I NTRODUCTION

Nowadays, smartphone is used more and more frequently. Smartphone contains diverse and powerful sensors. These sensors include GPS, microphones, cameras, light sensors, compasses, proximity sensors, and accelerometers. Smartphone has qualities of portability, unprecedented computing power, and ability to receive and send data via network, and nearly ubiquitous usage in our modern society, so smartphone establishes an exciting new area for data mining and human-computer interaction [3]. In this situation, the smartphone has evolved from simple computing and communications equipment into a set of portable personal intelligent terminal including computing, perception and communication. With more and more rapid development of context-related services, a variety of new human-computer interaction emerges [10]. In this paper we study application of the accelerometer sensor in smartphone to identify the state of motion activity a person is performing. This research is known as activity recognition.

User activity contains abundant contextual information. User activity reflects the current state and intension of individual motion in a short period of time; in the long term, user activity reflects the diverse characteristics of individual preferences and habits. Therefore, the recognition of user activity is an important foundation to provide personalized service. We achieve our activity recognition by utilizing machine learning algorithms under supervised condition. We collect our accelerometer data from 3 experimenters when they perform daily activity, then transform raw data into examples that can be classified. Each example is labeled with the certain type of activity during data collection stage. And activity recognition task is accomplished by standard machine learning algorithms. We choose Android-based smartphones because Android operating system has advantages of open-source, convenient for programming, and android operating system is the most popular system among smartphone system [9]. Since most smartphones and some music players have equipped tri-axial accelerometer, our research for activity recognition could be applied to other mobile equipment easily.

Accelerometer placed inside smartphones is used to support advanced game play and to enable automatic screen rotation [11]. Furthermore, it can also provide services that develop activity recognition. It’s convenient to get user’s activity data when a user carries smartphone in pocket during daily life [12]. So we can develop applications such as health monitor and fitness trainer. Health monitor can detect person’s seizure of diseases such as cardiovascular disease and hypertension by classifying person’s inappropriate activity [5]. Fitness trainer collects person’s movement information during exercise, and then analyzes person’s exercise indexes such as exercise time, calories expended per hour. Thus, fitness trainer guide person with good advice [2].

Accelerometer-based activity recognition is not new. A great deal of activity recognition applications has been developed. But some researchers use specific accelerometer-based equipment (belt or watch) to record user’s activity data [1]. Smartphone-based activity recognition has some advantages: Smartphone is much cheaper than specific

device, and smartphone can get accelerometer data without disturbing persons’ daily life. Specific accelerometer-based equipment needs to be tied on person’s body (wrist, arms and legs) and this situation results in bad user experience.

There are lots of smartphone-based applications. They only extract time-domain features from acceleration data as classification features. The time-domain features include average, standard deviation, average absolute difference, average resultant acceleration, binned distribution and correlation. Method above is easy to process and time-saving. But these features can’t describe states of motion accurately. In this paper, we propose a novel method to extract both time-domain and frequency-domain features from acceleration data. Results show that our method reaches higher accuracy than method which extracts only time-domain features on equal condition.

II.A CTIVITY R ECOGNITION G OAL

The goal of activity recognition is recognize a certain type of motion activity in daily life via machine learning algorithm [4]. This work contains three parts. First, we collect time-series accelerometer data by directing experimenters performing specified motion while carrying smartphone in their trousers pocket. Then we label data manually as different types of activities according to collection status. Six common activities are considered: standing, sitting, stair climbing (up and down), walking and jogging. Then we transform raw data to examples. Last, we draft a classification algorithm for these examples. Fig 2 shows accelerometer data graph for walking, jogging, sitting and climbing downstairs. These six activities are divided into two groups: motionless activities (standing and sitting) and motion activities (stair climbing, walking and jogging) [6]. There is notable distinction between motionless activities and motion activities. Due to gravity, value of y-axis will increase 9.8 when smartphone is upright.

III.E XPERIMENT M ETHODOLOG

In this part we present the methodology for creating the activity recognition models. We discuss the data collection process, the method for transforming the time-series data into examples, and the model induction process.

A.Data Description

We collected our test data by directing 3 experimenters (male: 2, female: 1) carrying an Android-based smartphone in their trousers pocket while performing the six activities described previously. We use Android application during this collection process. The application is installed at three different brand smartphone. Android OS versions vary from 3.5 to 4.0. Accelerometer data is transmitted to computer server from smartphone via internet. Sampling rate of accelerometer data is 20 Hz. The accelerometers placed in all of these smartphones are observed to work in similar manner.

B.Data Transform

Since we can’t deal with time-series accelerometer data directly during classification procedure, we need to transform raw data into examples. We use wavelet as a tool to get both time-domain and frequency-domain information from raw data. Wavelet is chosen because wavelet decomposition can cover the entire frequency domain with providing a complete mathematical description and wavelet transform can greatly reduce or eliminate correlations between different features by selecting the appropriate filter. The most common used wavelet transform in data transform process is DWT (Discrete Wavelet Transform). But DWT has a significant disadvantage: lacking of shift invariance, which means that small shifts in the input signal can cause major variations in the distribution of energy between DWT coefficients at different scales. In our research, motions including walking and jogging are sensitive to time-invariance. So we need to find a new form of wavelet.

Nick Kingsbury [8] discovers a more computationally efficient approach of discrete wavelet transform called dual-tree complex wavelet transform (DT-CWT) which can cover shortage of DWT. The new approach is that using binary tree based on DWT, one tree generates real transformation, and the other generates imaginary part. The theoretical basis is as follows: for the same layer of dual-tree, if the delay between filters from two trees is just a sampling interval, it’s convinced that during down-sampling by 2 processes, real tree captures exactly what imaginary tree lost during down-sampling by 2 processes. This method will eliminate effect of shift-variance. We use DT-CWT as our data transform tool. The procedure includes choosing appropriate transform level and generating filters, then generating features through DT-CWT filters.

DT-CWT feature has characteristics that it contains high redundancy. DT-CWT statistical information can solve high redundancy problem. Statistical information includes DT-CWT mean and variance. Statistical information can present signal characteristics of different scales which is based on multi-resolution analysis. It puts aside a large number of DT-CWT coefficients which needs to deal with directly. It focuses on universal quantity extracted from coefficient; secondly, statistical information has quality of a clear meaning, simple calculation, and non-sensitive to noise.

Since activity data indicates orientation of motion as well. In this paper, orientation features are extracted from acceleration data. We propose 3 angles to describe orientation features. Pitch angle, roll angle and azimuth are rotating respectively around X, Y, Z axis. We gained features in 4 steps below. In order to compare the different effects of DT-CWT feature, DT-CWT statistical information and orientation features, 3 types of features are shown in Fig.1 by our algorithm.

C.Experiments

Our activity recognition model is created from the labeled examples using the following WEKA machine learning algorithms: decision trees (J48 and Random Forest, RF), instance-based learning (IBk), and rule induction (J-Rip). Default settings are keeping for all machine learning methods except for IBk, where we set k=3 (IB3) instead of the default of 1.

Our machine learning model selects 3 types of feature as input: feature A, feature B, feature C. Feature A is raw coefficient of accelerometer data. Feature B is raw coefficient of accelerometer data and orientation information. Feature C is statistics information of Feature B. We used 10-fold cross validation as our evaluation method. In other words, activity examples in the data set were divided into 10 parts equally and randomly. One part was selected to serve as checkout data, while the remaining 9 parts were used for training the classifier. This procedure was performed 10 times while iterating the test set over the 10 partitions of the

The predictive accuracy for each of the four classification algorithms and different features is shown in Table I. Indication in first three rows is result of separately utilizing F A,F B and F C. Indication in fourth row is outcome by utilizing algorithm proposed by Gary M. Weiss and Jeffrey W. Lockhart in Fordham University. Their data transform method is using time-domain features such as average, standard deviation, average resultant acceleration, time between peaks and binned distribution [7].

Table I Predictive Accuracy of Activity Recognition Performance for

Different Features

Predictive Accuracy for different

features

Classification Algorithm

RF IB3J48J-Rip

F A0.760 0.738 0.674 0.674

F B0.818 0.787 0.742 0.743

F C0.847 0.860 0.812 0.784

Other Algorithm 0.759 0.684 0.691 0.702

Record in first line of Table I shows that random forest algorithm performs better than the other three algorithms when using feature A as input. It also indicates that our result performs better than Gary M. Weiss and Jeffrey W. Lockhart’s method when using random forest and IB3 as classification algorithm. But when using J48 and J-RIP as classification algorithm, our result is worse than Gary M. Weiss and Jeffrey W. Lockhart’s method. Record in second line shows that random forest algorithm performs better than the other three algorithms when using feature B as input. It indicates that our result performs better than Gary M. Weiss and Jeffrey W. Lockhart’s method when using all four classification algorithms as well. Then all accuracies are better than those using feature A as input. Record in second line shows that IB3 algorithm performs better than the other three algorithms when using feature C as input. All accuracies are better than those using feature B as input.

This result implies two pieces of information in the matter of feature selection: accelerometer data can’t be just treated as data flow since orientation information gives rise to better performance. And wavelet statistics information performs better than raw wavelet coefficient. Indication in table II also presents that our data transform algorithm which contains both DT-CWT statistical information and orientation information performs better than Gary M. Weiss and Jeffrey W. Lockhart’s algorithm, which implies that time-domain and frequency-domain features can cover most information in accelerometer data. And only using time-domain features can’t classify users’ motion activity very precisely.

Statistics number also shows that among 4 classification algorithms, random forest performs best when the input feature has high dimension. Because random forest is established with a random way, there are a lot of decision tree in random forest. But there is no correlation between each decision tree in random forest. So the performance of random forest becomes better and better when the number of decision trees increases, in other words, random forest is proper for high dimensional features. When input feature has low dimension, IB3 classification algorithm performs better than the other three algorithms. Because IB3 is instance-

based learning classification algorithm. It means that this kind of learning classification only saves information of train set, and then makes classification decision after all test data samples arrive. The idea of IB3 algorithm is that if a test sample is the most similar to 3 training samples of the same category in the feature space, then this sample belongs to this category mentioned previous. So IB3 algorithm suits for low-dimensional features.

V.C ONCLUSION AND F UTURE W ORK

In this paper we establish and evaluate a feature extraction method and a machine learning approach for achieving activity recognition, in a convenient manner, using only a smartphone placed in user’s pocket. We demonstrate that by using both time-domain and frequency-domain features as training feature, our classification accuracy can achieve a very high score. In this paper, we use DT-CWT wavelet transform to acquire time-domain and frequency-domain features. The results prove that DT-CWT transform can enhance classification performance. In addition, adding orientation information can help raise classification accuracy.

We intend to extend our activity recognition task in several patterns. We plan to add other activities and to utilize additional sensors such as GPS and gyroscope. Since motion activity varies greatly from individual to individual, we consider add another non-accelerative messages to get more precise accuracy. In consideration mentioned above, we tend to add user’s age, gender, body weight and ratio of motion activity’s duration to motionless activity’s duration per day.

R EFERENCES

[1]Fujimoto T, Nakajima H, Tsuchiya N, et al. Wearable human activity

recognition by electrocardiograph and accelerometer[C]//Multiple-

Valued Logic (ISMVL), 2013 IEEE 43rd International Symposium on. IEEE, 2013: 12-17. [2]Boerema S T, van Velsen L, Schaake L, et al. Optimal sensor

placement for measuring physical activity with a 3D

accelerometer[J]. Sensors, 2014, 14(2): 3188-3206.

[3]Anjum A, Ilyas M U. Activity recognition using smartphone

sensors[C]//Consumer Communications and Networking Conference

(CCNC), 2013 IEEE. IEEE, 2013: 914-919.

[4]Dernbach S, Das B, Krishnan N C, et al. Simple and complex activity

recognition through smart phones[C]//Intelligent Environments (IE),

2012 8th International Conference on. IEEE, 2012: 214-221.

[5]Viet V Q, Lee G, Choi D. Fall detection based on movement and

smart phone technology[C]//Computing and Communication

Technologies, Research, Innovation, and Vision for the Future

(RIVF), 2012 IEEE RIVF International Conference on. IEEE, 2012:

1-4.

[6]Weiss G M, Lockhart J W, Pulickal T T, et al. Actitracker: A

Smartphone-based Activity Recognition System for Improving Health

and Well-Being[J]. 2014.

[7]Weiss G M, Lockhart J W. The impact of personalization on

smartphone-based activity recognition[C]//AAAI Workshop on

Activity Context Representation: Techniques and Languages. 2012.

[8]Kingsbury N. Complex wavelets for shift invariant analysis and

filtering of signals[J]. Applied and computational harmonic analysis,

2001, 10(3): 234-253.

[9]Zhang S, McCullagh P, Nugent C, et al. Activity Monitoring Using a

Smart Phone's Accelerometer with Hierarchical Classification[C]//Intelligent Environments (IE), 2010 Sixth

International Conference on. IEEE, 2010: 158-163.

[10]Xiqun C, Xinmiao Y, Qixin S. Traffic Analysis Zone Based Urban

Activity Study with Aggregate Mobile Network Data[C]//Management and Service Science, 2009. MASS'09.

International Conference on. IEEE, 2009: 1-4.

[11]Li M, Wang H, Guo B, et al. Extraction of human social behavior

from mobile phone sensing[M]//Active Media Technology. Springer

Berlin Heidelberg, 2012: 63-72.

[12]McMillan J E R, Glisson W B, Bromby M. Investigating the increase

in mobile phone evidence in criminal activities[C]//System Sciences

(HICSS), 2013 46th Hawaii International Conference on. IEEE, 2013:

4900-4909.

Figure 2 Accelerometer data graph for activities (a) Walking (b) jogging (c) sitting (d) climbing downstairs