Skip to main content

The Bitter Truth of Benchmark Scores !

 

It's the golden age of smartphones and the consumers are becoming "smart" when purchasing one. One of the parameters that the younger generation looks out for is the benchmark scores of a smartphone. This article isn't just another one which just compares the scores of different smartphones. I mean common, people can read numbers, can't they ? Benchmarks are used to compare the performance of a phone with respect to its competitors. Higher numbers mean better performance. But, have you wondered how are these benchmark scores calculated ? What all stuff do they run during this benchmark process ? Does it really give useful information about the phone as such ? today, we will try to answer these questions in the simplest way possible. 

Benchmark scores are numbers that represent the performance of a smartphone based on various metrics. The metrics that are considered generally by most tech reviewers and consumers is the Single Core CPU and Multi Core CPU performance. In order to make sense of these numbers, we have to understand that these are not absolute numbers but relative numbers. Now to which chipset is the comparison done with ? In most cases the Intel Core i3 processor is used as the reference for which 1000 points is assigned. Now, if a new chipset performs better than the i3 processor then the score is higher than 1000 else it's lesser than the performance of the i3 processor. With that out of the way , let's go a bit deeper into what kind of stuff is run as part of the benchmark. Here, we take Geekbench benchmark application for reference. 

The Single Core CPU test considers 3 types of "Loads". "Loads" here mean different types of tests. The 3 Loads are - Cryptography, Integer and Floating Point. The weightage given to each of these is 5%, 30 % and 65% respectively. Why is the Floating point test given more weightage one might wonder ? The reason is in the modern day application, the CPU deals with applications that have majority operations of the type of Floating point. Floating point operations do not just involve fractional operations, but in terms of the processors it's used to do large precision operations. What that means is to do multiple 64 bits operations in one cycle. The more number of bits you process, the faster the CPU works. So, the CPU tries to convert a given operation into Floating point as often as possible, because it has dedicated Hardware to do the operation quickly. That's the reason why FPUs are given higher weightage. Cryptography operations are used typically when we use applications that demand certain level of "security". For example, if you would have noticed,nowadays the websites start with prefix " https ". "S" stand for secure in which encryption algorithm is run before sending or receiving any kind of data. Another application that makes use of this encryption is "Whatsapp". As the concerns for security and privacy are increasing, the weightage given to Cryptographic operations will gradually increase. So, let's say if the i3 processor was able to perform the benchmark in 10s each test respectively which means a total of 30 seconds for the entire test. Now, a new smartphone performs those same tests in 5 seconds each, that means it performed the entire test in 15 seconds which is twice as fast as the intel i3 processor. So, the benchmark score would be 2000 as it performs 2x faster than the base line.

Under the integer type loads, the applications such as text compression,image compression, navigation, HTML5, PDF reading, basic camera processing. Under the Floating Point Workloads stuff like gaming, sensor based processing, advanced image processing, face detection,  edge detection, image editing, HDR display, depth sensing, speech recognition, Machine Learning and AI. These cover the general usage of the applications what generally people use on daily basis. But, the important point is the usage differs from person to person. Consider a pro gamer, who uses his smartphone for most part of the day which means that the Floating Point operations would be done for 90% of the time. Remaining time would be distributed between Integer and Cryptography. On the other extreme, consider a 50 year old person who's not so tech savvy. In his case, he might use most part of the day scrolling through websites and occasional social media. In his case, the usage of cryptography and Integer loads are 90% of the usage. So, you see what I mean ? the benchmarks are done based on a statistical data gathered on the daily usage of different applications. Out of which only the major ones are tested on as part of the benchmark application. 

That's why many tech reviewers always say that take the Benchmark scores with pinch of salt because they cannot exactly tell how good the phone is. Moreover, in order to woo the consumers, certain manufacturers optimize their hardware and software just for these Benchmark applications to rake in high scores. The scores should just be used as indication to how the smartphone would perform under "CERTAIN" conditions only. The real test of the smartphone would be seen only when used daily. 

I hope most of the misconceptions are waded off for now and a new perspective is added to the way you view smartphones and purchase them. Let me know if you found this article helpful in the comment section !:)

Apple IPhone 12, Watch Series 6, iPad AIR Launch Event 

This Smartphone is a Chameleon !







OnePlust 8T/OnePlus 8T Pro What to Expect and Launch Date ?


Apple iPhone 12 Price Revealed And We're not Happy About it





















Comments

Popular posts from this blog

Don't think about Buying Tablet in 2020! They are going to be Dead!

Remember the time when the "mobile" phones were not "smart" yet and had this really compact form factor with durability such that they can only be destroyed by some natural calamity ? Yep, those times were not far from today ..just a decade ago and we didn't have to think about the phone falling off a cliff, or catching fire or trampled under a car or drowned into a sea! Then there was this highly delicate and expensive loner phone called the "iPhone" which required such extreme care, for you wouldn't be able to afford its maintenance! This phone was one of the first ones to start a trend of increasing the display size over the year and that led to hilarious memes as below Being mocked by their competition only strengthened their move and Apple went onto release the iPhone 11 Pro Max or Max Pro (whatever!) which was 6.5 inch long and was close enough to the prediction in the memes. This time it wasn't getting mocked but was

How to Pair Huawei Watch GT2 with Any Headphones within 1 min - the Perfect Hack

The Smartwatch and wearables industry has seen the biggest spike in 2019-2020 time frame. Starting off with Apple Airpods to may other brands trying to mimic the design and provide low cost alternatives the users have graciously welcomed each and every product in the line. There is a fan base for the neckband type, another base for the "TWS" (Truly Wireless) Earbuds and there's altogether separate base for the Wireless Headphones category. Then we saw an upsurge in the health monitoring wearable devices which was popularized by Fitbit and then in the last couple of years by Apple and Xiaomi. The current market has many players and there's no clear winner in terms of market share for the smartwatches / health monitoring wearables. Huawei has somehow missed the train and joined in late for the party. But as they say in Hindi - " Der aaye Durust Aaye .. "  which means - Came in Later But in a mint condition .. Huawei launched its first smartwatch the

Sony Xperia 1 II - the Lazarus of Smartphones

Well, well, well... Look who's back on the Street ! He's tall ..he's strong..he's Loaded ! That's exactly what the Xperia 1 II is all about ! Sony can well be termed as the "Lazarus"  for coming back from the dead - quite literally. It was standing against all the odds with likes of Samsung, Apple, One Plus in Premium and Xiaomi, Oppo, Vivo, Huawei in the Mid Segments, Sony had no where to land and with this particular model they have reaffirmed their faith in their Premium segment and its definitely Working ! I know the naming is weird (seriously who names their product "one" "two" ) Check out their official Video: Credits : Sony Youtube Channel Credits : NDTV Mind you , this is the first smartphone with a 21:9 4K HDR OLED 6.5" Display with 90 Hz Refresh Rate - let that sink in. Sony has used its experience in its Television Display tech and dropped it straight up on the Xperia 1 II. With the notch nicely hid