Benchmark scores are essentially the best method to gauge a device’s performance and that is something many people opt to measure hardware performance before actually buying a smartphone or PC, etc. Benchmark cheating is a term which is used by smartphone makers unfairly try to trick benchmark measuring apps and tools into disguising their not-so-good results into flamboyant and high-performance results. What’s the goal, you may ask? The goal is to achieve dominance over competitors.
Although there are several smartphones that have been alleged for benchmark cheating such as Samsung Galaxy S4 (2013), Samsung Galaxy Note 3 while brands like Meizu tweaked the results using different performance modes to showcases higher results. Recently, a report was published by Cutress which showcased how Huawei has practiced benchmark cheating on its 2018 Huawei P20, Honor Play, and Huawei P20 Pro.
As said, Huawei has equipped the 2018 Huawei and Honor smartphones with benchmark detection mechanism that detects if any benchmark tool or app is being used which is when it increases the power limit of the SoC to provide higher performance scores when compared to other apps. Huawei does the same with the help of whitelisting few applications as reported by AnandTech. Doing so decreases the efficiency of the SoC while increasing the power consumption and reduced battery life to name a few.
When tested under normal conditions, the tests keeping the benchmark detection mechanism ON and OFF show a contrasting difference. When questioned about the same with Huawei, the Chinese smartphone manufacturer stated that benchmark cheating is common in China where smartphone manufacturers tweak the characteristics to get higher scores and mislead users who opt for a false benchmark score while buying a smartphone. Huawei further added that the maker is finding it difficult to face off against other Chinese manufacturers flaunting their unrealistic scores and that is why Huawei has a goal to standardize the benchmarks scores.
When AnandTech further tested the GPU benchmarks to compare the publicly available benchmarks, it found out that all three smartphones i.e. Honor Play, Huawei P20, and P20 Pro perform identically with HiSilicon Kirin 970 SoC set on a high power mode, however, the performance of the said smartphones literally depends on the thermal limits which are indeed based on various aspects and it is different for all the three smartphones.
Usually, smartphones manufacturers would resort to higher SoC frequencies or lock it down to generate higher results, however, Huawei has opted for a different technique wherein it actually whitelist few apps including benchmark tools under use case scenario where the SoC would actually deliver higher performance as advertised for the whitelisted apps. On the contrary, the phone’s performance depends on the thermal limitations of the hardware in real work which is perhaps throttled by the SoC.
In another official statement by Huawei, it credited technologies such as artificial intelligence which comes bundled with Kirin 970 SoC that optimizes performances of NPU, GPU, and CPU among other components to provide a smooth and flawless user experience rather than providing the best-in-class smartphone benchmarks which aren’t something Huawei is keen to elevate since it depends mostly on user experiences.
When AI works with SoC, it optimizes the power consumption, processing power in such a way that AI will deliver maximum hardware capabilities to graphics intensive games or apps while those apps that require less energy gets the required resources from the SoC as per requirements.
As of now, UL which has acquired benchmark firms such as Futuremark that owns 3DMark, PCMark, has delisted Huawei Nova 3, Huawei P20 Pro, and Honor Play from 3DMark website after finding varying results on both private and public app. It added further that although Huawei has an intention to use AI as a way to allocate resources, however, it forces the smartphone to enter into the Performance Mode whenever it detects benchmarking apps which are why, it had to delist it although Huawei claims that it had discussions with UL and has temporarily removed the results until all users get access to the Performance Mode on their Huawei and Honor smartphones.
