Find in Library

This work develops a generalizable neural network, SENSORNET, for sensor feature learning across various applications. The primary challenge addressed is the poor portability of pretrained neural networks to new applications with limited sensor data. To solve this challenge, we design <span style="font-variant: small-caps;">SensorNet</span>, which integrates the flexibility of self-attention with multi-scale feature locality of convolution. Moreover, we invent patch-wise self-attention with stacked multi-heads to enrich the sensor feature representation. <span style="font-variant: small-caps;">SensorNet</span> is generalizable to pervasive applications with any number of sensor inputs, and is much smaller than the state-of-the-art self-attention and convolution hybrid baseline (<inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>0.83</mn></mrow></semantics></math></inline-formula> M vs. <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>3.87</mn></mrow></semantics></math></inline-formula> M parameters) with similar performance. The experimental results show that <span style="font-variant: small-caps;">SensorNet</span> is able to achieve state-of-the-art performance compared with the top five models on a competition activity recognition dataset (SHL’18). Moreover, pretrained <span style="font-variant: small-caps;">SensorNet</span> in a large inertial measurement unit (IMU) dataset can be fine-tuned to achieve the best accuracy on a much smaller IMU dataset (up to <inline-formula><math xmlns="http://www.w3.org/1998/Math/MathML" display="inline"><semantics><mrow><mn>5</mn><mo>%</mo></mrow></semantics></math></inline-formula> improvement in WISDM) and to achieve the state-of-the-art performance on an EEG dataset (SLEEP-EDF-20), showing the strong generalizability of our approach.