During our testing programme of 2020/2021, we encountered a strange bug on a brand new flagship phone, the OnePlus 8T. This bug seemed to happen at random, causing the Carv app to stop detecting turns whilst skiing. It was, and still is, one of the larger headaches of the season for us.
The bug wouldn’t happen consistently and there was no pattern to explain its occurrence. This meant that reproducing the bug was very difficult.
Over time, we noticed a trend: the worst-affected devices all have the new (as of 2020/2021) SnapDragon 865 SoC (System on a Chip). These include Samsung’s S20 range, OnePlus’ 8 range, LG’s V60 ThinQ, Sony’s Xperia 1 & 5 (mk.2) and Xiaomi’s Mi 10 range.
What does the bug do?
There are two parts to this issue, which appear to the skier in slightly different ways.
1. Hearing “Error, bluetooth is turned off”
The first part of this bug is a crash. The ‘bluetooth stack’ on the phone crashes, and to remedy this crash, the device turns bluetooth off and on again.
In the Carv app, we have a spoken warning that says, “Error, bluetooth is turned off”. This warning is triggered here because the phone has turned off bluetooth to remedy the crash.
2. Time Travel
On the Carv session recording screen, there’s a big clock timer, to indicate the length of time you have been recording. The second part of this bug would cause the clock to jump ahead by approximately 50 days, or 1193 hours. When this happens, the app would stop detecting skiing turns, rendering the app fairly useless until the skier restarts their session.
Why do these things happen?
The cause of the bluetooth stack crash is an unknown ‘hardware error’, meaning something went wrong with the bluetooth chip or the chip's firmware. The logs indicate a specific error code, but this error code isn’t documented for public consumption.
The cause of the clock jumping ahead is corrupted bluetooth data. You can read a technical explanation of why this happens at the bottom of this post.
Can you fix it?
Sadly, there’s not much we can do until phone manufacturers fix the issue.
Both the crash and the corruption happen outside the responsibility of the Carv app.
The bluetooth crash affects the whole device and isn’t caused by anything our app is doing.
The bluetooth corruption happens before our app gets hold of the data and the logic that handles the clock rollover works properly when the input data isn’t corrupted.
What happens now?
Regarding the crash
We have reported our findings for the bluetooth crash to the most affected manufacturers. These include:
Samsung - via public forum post and private support email conversation with their support team
OnePlus - via public forum post and private support chat with their support team
LG - via email support
It now falls to these manufacturers to debug and fix the cause of the crash.
As of writing this blog (Mon 26th April 2021), Samsung has suggested that the crash should be fixed in their latest software updates, but only for phones running android 11.
OnePlus continues to work on it.
Regarding the corruption
Gathering the required information to submit a bug report for the bluetooth corruption is a little harder as corruption occurs infrequently, under unknown circumstances, and requires specialised equipment to diagnose.
However, we have managed to reproduce the corruption on a OnePlus 8T and have sent all supporting information to OnePlus to debug this issue. We hope they can provide an update soon.
Since Samsung fixed the crash, we haven’t been able to reproduce the corruption. This backs up the theory that the two issues are linked and probably have the same root cause.
A technical explanation (jargon alert!)
If you’re interested in the ‘why’ more than the ‘what’, here’s a brief technical explanation of why the corruption part of this bug manifests as it does.
In the Carv app, there is a piece of logic that detects when a bluetooth packet timestamp is before the previous packet's timestamp, and assumes the timestamp has ‘rolled over’. A timestamp ‘rolling over’ means that it has reached the maximum value that can be stored in its variable and has returned to 0, meaning we need to modify the timestamp epoch to take this into account.
In practice, this adds the maximum value of an unsigned 32 bit integer divided by 1000 (to get the timestamp in seconds) to the epoch. So, 4,294,967,295 / 1000 = 4,294,967.295, and 4,294,967 seconds = ~1193 hours.
Because of this, when bluetooth data is corrupted and the timestamp portion of the packet is incorrect (corrupted and appearing to be lower than the previous timestamp), the app will think the timestamp has rolled over and show ~1193 hours.
Usually, this will only happen on one of the two trackers, meaning that the two trackers are now in different time frames and as such, turns will not be detected.
Bluetooth has mechanisms built in such as a CRC (cyclic redundancy check), that should prevent such corruptions from happening over the air. This means any data corruptions are usually bugs in the phone's bluetooth firmware.
A final note
It isn't only Snapdragon 865 based phones that have this corruption issue, we have noticed the Motorola G8 power, and Samsung A5 also exhibit this corruption bug very frequently and as such we have had to mark them as 'incompatible' on our Compatibility Testing support page.