Domain adaptive methods for device diversity in indoor localization

Liu Jing Liu Nan Pan Zhiwen You Xiaohu

(National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China)

AbstractTo solve the problem of variations in radio frequency characteristics among different devices, transfer learning is applied to transform device diversity to domain adaptation in the indoor localization algorithm. A robust indoor localization algorithm based on the aligned fingerprints and ensemble learning called correlation alignment for localization (CALoc) is proposed with low computational complexity. The second-order statistical properties of fingerprints in the offline and online phase are needed to be aligned. The real-time online calibration method mitigates the impact of device heterogeneity largely. Without any time-consuming deep learning retraining process, CALoc online only needs 0.11 s. The effectiveness and efficiency of CALoc are verified by realistic experiments. The results show that compared to the traditional algorithms, a significant performance gain is achieved and that it achieves better positioning accuracy with a 19% improvement.

Key wordswireless local area networks; indoor localization; fingerprinting; device diversity; transfer learning; correlation alignment

When online terminals are different from those used to build the fingerprint database in the off-line phase, the signal changes are caused by different transmission power levels. The difference of received signal strengths(RSSs) from different devices exceeds 25 dB even at the same position[1], which will transfer the risk of mapping to different physical locations.

Different wireless chipsets, wireless antennas, hardware drivers, encapsulating materials and even operating system (OS) will lead to different RSSs. Wi-Fi chipsets on different terminals have different sensitivities to different APs and channels[2]. Since the antenna can be installed in different positions on the phone, the RSSs in different directions can be different. Wireless devices in the same reference point (RP) will not read equal RSSs due to heterogeneous reception characteristics of network interface controllers (NICs)[3]. Such heterogeneity can be attributed to a lack consistency in various standards among hardware vendors[4]. Two different devices using the same wireless network card can also receive different RSSs when the rear case materials are different[5]. Moreover, the detection rate and amounts of APs are likely to differ with different OSs[6]. Given different OSs, heterogeneous spectrums of the same terminal may be spawned.

To handle the device dependency, several studies have proposed methods to improve the robustness as below. Differences can be reduced by the linear relationship of different devices with massive manual calibration[7]. Signal strength difference (SSD) selects RSSs from one AP and each of other RSSs is deducted to eliminate the gain of antenna[8-9]. The RSS ratio is similar in ignoring other factors such as wireless network cards, antennas and OSs[10]. The complexity and cost of the expectation maximization (EM) positioning system is much higher and can be affected by the added noise and local optimum[11]. The relative rank of APs may be the same, resulting in low granularity of localization[12]. Filtering incorrect fingerprints of labor-intensive crowdsourcing is needed to avoid low stability due to heterogeneous devices or wrong positions[13]. Cosine similarity based on KNN is proposed to replace the Euclidean distance, which is limited in application scope[14]. Source and target domains can be projected into the low-dimensional manifolds to bridge the gap of the subspace while the first k-dimensional eigenvectors cannot represent all features[15]. Multiple devices are treated as multi-task learning, which requires similar potential features[16]. Source data is extrapolated to target data with inflexible symmetric transformation, ignoring the difference between the two domains[17]. The channel state information (CSI) shows frequency diversity and spatial diversity properties while CSI can only be collected by mobiles with an impractical Intel 5300 network interface card (NIC)[18].

It is expensive to recollect RSSs for calibrating all kinds of devices. A more practical method would be integrating RSS of unknown devices with fingerprints collected before on other devices adaptively. To implement device independent localization, we proposed an easy practical method named correlation alignment for localization(CALoc).

1 Preliminaries

The RSSs are not compatible with the off-line radio-map in practice, leading to accuracy degradation and limiting the applicability of the Wi-Fi fingerprinting localization system. In order to overcome the heterogeneous problem and avoid the off-line calibration effort, the proposed CALoc system is based on domain adaptation online. First, we study the effect of device dependency. Secondly, we propose correlation alignment (CORAL) to align across diverse devices.

1.1 Device diversity

To illustrate the signal variation over different devices, we empirically study different signal sensing capacities and yield different data distributions.

Traditionally, RSS difference can be generally considered additive deviations to RSSs[19]. In Fig.1, we collect instantaneous RSSs from AP c4:ca:d9:7b:db:60 for 10 min by HUAWEI 5X, MI 5, MI 5s, ZTE Nubia Z17 and MEIZU Pro6. Different terminals collect RSSs from the same AP at the same place. RSS collected by M5s is distributed from -52 to -46 dBm and the signal distribution of the Z17 device is -64 to -57 dBm. The RSS difference is more than 20 dB. We introduce the impact of WLAN chip on the statistical characteristics of RSS. According to IEEE 802.11, the range of the received signal strength indicator(RSSI) is defined between 0 and 255. The nonlinear map between the actual RF energy and RSSI is different and vendors will not expose its mapping relationship in detail. RSSs from the same AP at the same location can be roughly recovered from another by translation or scaling, which verifies the effect of device heterogeneity.

Fig.1 Difference of RSSs collected by different terminals

Moreover, the RSSs of different devices show different noise statistics, and their distributions are also significantly different. The main difference in RSSs is not only the linear difference between RSSs, but also the local deviation and distribution of RSSs. Fig.2 shows the difference of RSSs in the histogram distribution. Such heterogeneity in devices can be attributed to a lack of consistency in the various standards among hardware vendors, such as differences in the hardware and software used for different devices. The inability of RSS to match heterogeneous devices clearly reveals the problem of device diversity. It is easy to misjudge the location when matching the off-line database with the fingerprints using Euclidean distance.

Fig.2 Different distributions of RSSs collected by different terminals

Thus, the design of localization systems, which can inter-operate across different types of devices, is challenged by variations in RF characteristics between various devices.

1.2 Transfer learning

Machine learning is unable to work well when the distributions of training and test sets are different, which is easily affected by the shift of the learning domain. Moreover, supervised learning only performs well with a large amount of labeled training data. Many domain adaptive algorithms based on the labeled data easily compensate for the performance degradation caused by domain movement. However, unsupervised learning is necessary as labeled data is not easy to obtain in reality.

CORAL[20] align the second-order statistics of the source and target domain, which can confuse the source domain and target domain. Specifically, CORAL reconstruct the source characteristics with the covariance of the source features. Based on the covariance of the source domain and target domain, features in the source domain can be whitened with linear transformation to fill the gap between the domains. Although deep learning is widely used, a large number of parameters, such as the number of hidden neurons, need to be adjusted to apply them in new environments, which is costly.

In Fig.3, the rhombus represents the target domain and the dot represents the source domain. In the leftmost subgraph, we normalize the source and target domains with the Z-score normalization.

Fig.3 Introduction of CORAL in domain adaptation

The middle sub-graph illustrates that the source domain and the target domain are fused by whitening source features to remove the correlation of source features. As the rightmost sub-graph shows, the distributions of the source and target features can be well aligned when the target correlation is added to the source features. The classifier trained on the target domain can be expected to perform well in the adjusted source domain.

The target training data is given as RSST={rss1,…,rssm}, where rssiRn, and the source test data is represented as RS={r1,r2…,rk}, where riRn. Both the source domain and the target domain are characterized by n dimensional vectors. The mean and covariance matrices of the eigenvectors of these two domains are denoted as μS, μT and CS, CT. After normalization, μS=μT=0, while CSCT.

In order to reduce the gap between the second-order statistical characteristics of the source domain and the target domain, namely, the distance between CS and CT, the distance between the changed source domain features and the target feature covariance needs to be minimized. We choose a linear transformation A to the source features. The objective function is given by

(1)

where represents the covariance matrix of the transformed source domain features.

If the rank of the target domain is less than that of the source domain, the linear transformation A can be used as the alignment transformation to realize

Theorem 1[21] If the two real matrices Y and X with the rank of rY and rX satisfy the condition rXrY, we can do the singular value decomposition (SVD) of matrix Y: Y=UYΣYVY, where UY[1:r], ΣY[1:r], VY[1:r] are the left singular vector of Y, the largest r singular values and the corresponding right singular vector. Then, the optimal solution of

The linear transformation A does not increase the rank of ATCSA, namely, As the covariance matrix is symmetrical, it is easy to derive that CS=USΣSVS and CT=UTΣTVT with conduction of SVD on CS and CT. Since the rank of the covariance matrix of the source domain and the target domain is uncertain, the following two cases are described.

If rCS>rCT, the optimal solution is

(2)

where r=rCT.

If rCSrCT, according to Theorem 1, the optimal solution is

(3)

where r=rCS.

In conclusion, the optimal solution is

(4)

where r=min(rCS,rCT).

(5)

Suppose that the optimal solution of linear transformation matrix A is given by

(6)

In the above formula, the first part is the process of aligning the source data with the target covariance.

CORAL does not simply normalize features of source and target domains, but also aligns different distributions of these two domains. As correlation alignment only changes source features, it can be applied to any basic classifier. In this paper, we use ensemble learning to estimate the location. It is efficient and simple to adapt to the target data without subspace projection and dimension selection.

2 Localization Algorithm Architecture

The CALoc system is based on the Wi-Fi RSS fingerprint database, as shown in Fig.4. The labelled data collected by Device 1 is divided into training, validation and testing data. The proposed approach is learnt on training data with performance checks on validation sets. As usual, the final accuracy of the indoor positioning system can be predicted based on the testing sets collected by Device 2.

In this paper, the proposed CALoc algorithm adds an intermediate phase of online adjustment phase between the off-line phase and the online phase. In the extra added phase, we firstly calculate the test covariance and whiten the correlation of the source features with the inverse square root of the test covariance This asymmetric transformation aims to bridge the two device domains, so that the method can reduce the position error caused by the hardware difference between the training equipment and the test equipment. Here, the online adjustment process is introduced, as shown in the adjustment process of Algorithm 1.

After the adjustment, the trained ensemble learning system can be trained on the adjusted test features and predict the actual location. Because the online adjusted alignment changes only the characteristics, it can be applied to any base Wi-Fi localization system. Usually, the transformation takes 0.11 s, which is less than 1 min.

Fig.4 Localization Algorithm Architecture

Thus, the algorithm is extremely advantageous when the test domain changes rapidly.

Algorithm 1 Domain adaptation for heterogeneous devices

Require: Received signal strengths from , the labelled positions (xi,yi).

Ensure:The predicted positions(xi,yi).

while training do

Normalize RSS data and divide fingerprints with commonly used 10-fold cross-validation.

Construct ensemble learning with training data in the off-line phase to derive the map between fingerprints and locations. Then, we can save parameters of the trained model.

Calculate the train covariance and save it.

end while

while adjusting do

Whiten test data collected by unknown devices to remove feature correlations of source data in the online adjustment phase.

Align whitened data with the train covariance.

end while

while testing do

Localize with adjusted test data by the trained model in the online phase.

end while

3 Implementation and Localization Performance

In this section, the experiment environment is first briefly introduced, and then we analyze the performance of the proposed positioning algorithm.

3.1 Description of the experiment

Extensive experiments were conducted in the corridor on the second floor of a multi-functional lab, covering an area of 25 m×3 m. These fingerprints are evenly distributed in the area to be tested, with an interval of 1.6 m. In the off-line phase, fingerprints at each RP are collected for 2 min, while in the online phase, fingerprints at each RP are collected for 1 min. As the RSS collecting rate is 1 sample/s, we can obtain 120 training samples and 60 test samples at each reference point. In addition, we collected data at the same RP for one week, which results in large training and test sets. The performance was evaluated with 3 heterogeneous devices, i.e., MEIZU Pro6, HUAWEI Honor8 and HUAWEI 6X. To avoid the effects of different directions of the device, all devices are facing the same direction when RSSs are collected at the same time. Due to the different sensing capabilities of different devices, the number of APs received by each device is different, and only the common MAC features at both online and off-line stage can be selected as candidate features.

These three terminals can be combined into nine domain shift environments to verify the role of the proposed method. For instance, 6X-H8 means that the off-line radio-map is created by 6X but the positioning device is Honor8. The training and testing data are randomly sampled from the stored database.

3.2 Localization performance

We evaluate the performance of CALoc on the positioning of device diversities and compare it with direct localization, SMN and TKL.

When the same device is used off-line and online, we can obtain the optimal solution and the highest positioning accuracy. Fig.5 illustrates that better positioning performance occurs with the same device. In this case, the average localization error is 2.24 m. The changed test device without any pre-treatment can cause poor positioning performance, and the mean positioning error is 3.27 m if the test device is honor8 and the mean positioning error is 3.30 m if the test device is Pro6. Hardware variations significantly degrade the localization accuracy.

SMN reconstructs relative fingerprints to compensate for the shift effect caused by device diversity. Particularly,the RSSs at each RP need to minus the average RSS of all APs in the acceptance range. Then, the deviation of the RSS distribution caused by heterogeneities can be made up by removing the spatial mean. The TKL algorithm aims to learn how to extract potential features of source and target domains, then the source and target data can be matched in general Hilbert space. With the domain invariant kernel, the objective function minimizes the distance between the distribution of the source and target domains in Hilbert space.

Fig.5 Impact of terminal heterogeneities on localization

As shown in Fig.6, the proposed CALoc approach achieves better average performance across all nine domain shifts compared with the SMN algorithm and TKL algorithm. When the off-line device is honor8 and the online device is Pro6, the average positioning accuracy of CALoc under heterogeneous positioning decreases to 2.29 m while the average positioning accuracy is 3.00 m without any pretreatment. In this case, CALoc improves the positioning performance by 20%, while SMN and TKL do not work well. Thus, CALoc has an obvious role in removing the influence of equipment heterogeneity.

Fig.6 Localization with Pro6 from fingerprints collected by honor8

It is necessary to evaluate whether these algorithms produce side effects on localization under homogeneous devices. In Fig.7, when we use the same device to localize, the accuracy is raised by 10% with CALoc, higher than localization with SMN, which is raised by 5%. However, TKL decreases the performance of positioning with the same devices. When the testing and training data are both measured from the same device, the performance of TKL and SMN is clearly below CALoc. It illustrates that the improvement of SMN and TKL may be gained at the expense of discriminating information related to the same device.

Fig.7 Effect of pretreatment on localization performance with the homogeneous devices

In detail, Tab.1 records the good performance of CALoc under other domain shifts. Sometimes, the SMN algorithm cannot mitigate the impact of heterogeneity on the positioning performance, which is almost or even slightly inferior to the positioning effect without any processing operation. This is because the actual hardware dependence of device diversity is not only related to the antenna gain of the receiving terminals, but also closely related to OS or installation materials. Since the SMN algorithm only considers the antenna gain as the main influencing factor, its performance is inferior. Moreover, a large variance and small dimension are the disadvantages of SMN fingerprints which can lead to an unstable localization performance. For TKL, only the top k eigenvectors of the source and target covariance matrices can be aligned with the burden of subspace dimensionality selection. Moreover, TKL usually applies the same transformation to both the source and target domain. These symmetric transformation ignores the difference of the distribution of the source and target domain. Asymmetric transformation such as CALoc tries to connect two domains in a simple way. Although CALoc performs slightly better than TKL, CALoc online only needs 0.11 s while TKL needs 0.49 s for positioning from the perspective of time. In terms of performance, TKL can effectively alleviate the damage to localization accuracy caused by diversity, but the performance improvement is unstable. CALoc achieves better positioning accuracy with performance optimization ranging from 11% to 28%. It can be seen that CALoc can be adapted effectively to the second-order statistical characteristics.

Tab.1 Positioning performance with domain shift m

DomainshiftAverage localization performanceNoneWith TKLWith SMNWith CALoc6X-6X2.243.162.502.106X-H83.272.703.152.746X-P63.302.332.802.72H8-H82.513.102.332.30H8-6X2.742.352.222.27H8-P63.002.373.282.29P6-P61.823.192.001.58P6-6X2.252.101.911.67P6-H82.462.143.112.20

4 Conclusion

The training and test data are usually independent identically distributed in the indoor location problem. However, this assumption is rarely true in practice when the online positioning terminal and fingerprint terminal are inconsistent. To compensate for device diversity, we put forward domain adaptation before online positioning. Without the need of labelled data, the second-order statistical properties of the training and test data are aligned to minimize domain shift. The only computation of the efficient and easy CALoc method is aligning the whitened test features with the covariance of the training data. Extensive experiments demonstrate the superiority of the method.

References

[1]Zhang L Y, Ma L, Xu Y B, et al. Linear regression algorithm against device diversity for indoor WLAN localization system[C]// 2017 IEEE Global Communications Conference. Singapore, 2017. DOI:10.1109/glocom.2017.8254137.

[2]Mahtab Hossain A K M, Jin Y, Soh W S, et al. SSD: A robust RF location fingerprint addressing mobile devices’ heterogeneity[J]. IEEE Transactions on Mobile Computing, 2013, 12(1): 65-77. DOI:10.1109/tmc.2011.243.

[3]Sadiq S J. Device transparent rss-based indoor localization[D]. Toronto, Canada: University of Toronto, 2015.

[4]Jimenez E, Wei R Z. Indoor localization of ubiquitous heterogeneous devices[C]//Proceedings of the 2013 IEEE 17th International Conference on Computer Supported Cooperative Work in Design. Whistler, BC, Canada, 2013. DOI:10.1109/cscwd.2013.6581045.

[5]Wang Q. Kernel learning and applications in wireless localization [D]. New Jersey, USA: Rutgers University, 2016.

[6]Brouwers N, Zuniga M, Langendoen K. Incremental Wi-Fi scanning for energy-efficient localization[C]//2014 IEEE International Conference on Pervasive Computing and Communications. Budapest, Hungary, 2014. DOI:10.1109/percom.2014.6813956.

[7]Kjrgaard M B. Indoor location fingerprinting with heterogeneous clients[J]. Pervasive and Mobile Computing, 2011, 7(1): 31-43. DOI:10.1016/j.pmcj.2010.04.005.

[8]Laoudias C, Kolios P, Panayiotou C. Differential signal strength fingerprinting revisited[C]//2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN). Busan, South Korea, 2014. DOI:10.1109/ipin.2014.7275465.

[9]Zou H, Huang B Q, Lu X X, et al. Standardizing location fingerprints across heterogeneous mobile devices for indoor localization[C]//2016 IEEE Wireless Communications and Networking Conference. Doha, Qatar, 2016. DOI:10.1109/wcnc.2016.7564800.

[10]Cheng W, Tan K, Omwando V, et al. RSS-ratio for enhancing performance of RSS-based applications[C]// Proceedings IEEE INFOCOM. Turin, Italy, 2013: 3075-3083.

[11]Li L Q, Shen G B, Zhao C S, et al. Experiencing and handling the diversity in data density and environmental locality in an indoor positioning service[C]//Proceedings of the 20th Annual International Conference on Mobile Computing and Networking. Maui, Hawaii, USA, 2014: 459-470. DOI:10.1145/2639108.2639118.

[12]Kim Y, Shin H, Cha H. Smartphone-based Wi-Fi pedestrian-tracking system tolerating the RSS variance problem[C]//2012 IEEE International Conference on Pervasive Computing and Communications. Lugano, Switzerland, 2012:11-19. DOI:10.1109/percom.2012.6199844.

[13]Li Y, Williams S, Moran B, et al. A probabilistic indoor localization system for heterogeneous devices[J]. IEEE Sensors Journal, 2019, 19(16): 6822-6832. DOI:10.1109/jsen.2019.2911707.

[14]Han S, Zhao C, Meng W X, et al. Cosine similarity based fingerprinting algorithm in WLAN indoor positioning against device diversity[C]//2015 IEEE International Conference on Communications. London, UK, 2015:2710-2714. DOI:10.1109/icc.2015.7248735.

[15]Sun Z, Chen Y Q, Qi J, et al. Adaptive localization through transfer learning in indoor Wi-Fi environment[C]//2008 Seventh International Conference on Machine Learning and Applications. San Diego, CA, USA, 2008: 331-336. DOI:10.1109/icmla.2008.53.

[16]Pan S J L, Zheng V W C, Yang Q, et al. Transfer learning for wifi-based indoor localization[R]. Palo Alto, USA: Association for the Advancement of Artificial Intelligence (AAAI) Workshop, 2008.

[17]Zou H, Zhou Y X, Jiang H, et al. Adaptive localization in dynamic indoor environments by transfer kernel learning[C]//2017 IEEE Wireless Communications and Networking Conference (WCNC). San Francisco, CA, USA, 2017. DOI:10.1109/wcnc.2017.7925444.

[18]Sanam T F, Godrich H. An improved CSI based device free indoor localization using machine learning based classification approach[C]//2018 26th European Signal Processing Conference (EUSIPCO). Rome, Italy, 2018. DOI:10.23919/eusipco.2018.8553394.

[19]Chen L H, Wu E H K, Jin M H, et al. Homogeneous features utilization to address the device heterogeneity problem in fingerprint localization[J]. IEEE Sensors Journal, 2014, 14(4): 998-1005. DOI:10.1109/jsen.2013.2290736.

[20]Sun B, Feng J, Saenko K. Return of frustratingly easy domain adaptation[C]//Thirtieth AAAI Conference on Artificial Intelligence. Phoenix, USA, 2016, 6:1-8.

[21]Cai J F, Candès E J, Shen Z. A singular value thresholding algorithm for matrix completion[J]. SIAM Journal on Optimization, 2010, 20(4):1956-1982. DOI:10.1137/080738970.

室内定位中设备异构性的域自适应方法

刘 静 刘 楠 潘志文 尤肖虎

(东南大学移动通信国家重点实验室,南京 210096)

摘要:为解决不同终端之间射频特性的变化问题,借助迁移学习将室内定位系统的设备异构性问题转化为领域适应性问题,提出了一种复杂度低的基于相关对齐和集成学习的室内定位算法-相关对齐定位(CALoc).该算法只需要将实时指纹与离线指纹的二阶统计特性进行对齐.这种实时定位的在线校准方法很大程度上消除了在线指纹库与离线指纹库的设备异构性.该算法无需任何耗时的深度学习重新训练过程,在线定位只需要0.11 s.实际场景的实验结果表明,CALoc与其他传统算法相比取得了显著的性能提高,定位精度平均优化提高了19%.

关键词:无线局域网; 室内定位; 指纹; 设备异构性; 迁移学习; 相关对齐

DOI:10.3969/j.issn.1003-7985.2019.04.004

Received 2019-04-29,Revised 2019-10-15.

Biographies:Liu Jing (1994—),female,graduate; Liu Nan(corresponding author), female, doctor, professor, nanliu@seu.edu.cn.

Foundation items:The National Key Research and Development Program of China (No.2018YFB1802400), the National Natural Science Foundation of China(No.61571123), the Research Fund of National Mobile Communications Research Laboratory, Southeast University(No.2020A03).

CitationLiu Jing, Liu Nan, Pan Zhiwen, et al.Domain adaptive methods for device diversity in indoor localization [J].Journal of Southeast University (English Edition),2019,35(4):424-430.DOI:10.3969/j.issn.1003-7985.2019.04.004.

中图分类号:TN929.5