Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

In AIMark, the Mi9 offers the needed library support from Qualcomm for the benchmark to work. In terms of performance, the device ranks high, although it’s just slightly edged out by the rest of the Snapdragon 855 devices.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

The CPU results in AI Benchmark are relatively mixed in terms of their positioning for the Mi9. In some of them, the Mi9 falls in amongst the last S855 devices, while in others, it more in the middle of the pack. Overall, this is just a matter of how the scheduler and DVFS is tuned on the Mi9. It’s to be noted that the differences between the devices here is very minor, with only a 10% difference between the best and worst Snapdragon 855 devices.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

In the INT8 tests which are accelerated on the SoC’s Hexagon DSP, we’re seeing the Mi9 lands in the middle of the pack again.

On the last “Cartoons” test running VGG-19, we’re seeing that the Mi9 is falling behind the rest of the pack alongside the LG G8. The reason for this is likely that the device is shipping with older NNAPI drivers than the rest of the Snapdragon 855 phones. The firmware we’ve tested this on was the latest at the time of testing in early September.

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

In the FP16 tests the Mi9 again ends up middle of the pack.

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

Finally, the FP32 test running on the GPU ends up as the last Snapdragon 855 device in the pack, although again the differences here between the devices are very small, and it’s still performing vastly better than any other non-Snapdragon device.

System Performance GPU Performance
POST A COMMENT

96 Comments

View All Comments

  • Redmyth79 - Friday, September 13, 2019 - link

    I'm going to call Bullshit on your scores because I have the Note 9, S9+ and the Xiaomi Mi 9 EU Global ROM and my scores including FPS and a lot of things you have posted here do not come close to the performance I'm getting.
    My Mi 9 best every phone you have up there in most every one of those test.
    It lost to the ROG 2 and Black Shark 2pro in a couple area's but very limited. Matter fact though, what makes anybody with a ounce of common sense know that this testing is completely fabricated bullshit is the fact that the S10+ is up top or near the top in a lot of spots because the S10+ has the worst testing of just about any 855 chipped phone. Furthermore you have the S9+ rated higher in certain test that I personally ran and there are official results from those very testing sites saying your results are off by more then a 1000 points in some areas with the S9+ even.
    I don't get the sham going on here but it just ruined any credibility this site ever had!
    Reply
  • NXTwoThou - Friday, September 13, 2019 - link

    Are you still using the stock rom? I'm pretty sure AnandTech didn't bother getting the latest greatest(and probably hasn't had it long enough to get unlocked or xiaomi.eu). I know my phone had a huge performance difference after a few updates. Reply
  • Andrei Frumusanu - Friday, September 13, 2019 - link

    It's running the latest OTA on a commercial EU unit, build date July 9th. The phone's been out long enough that didn't go finishing for nonpublic firmwares. Reply
  • Redmyth79 - Friday, September 13, 2019 - link

    What update? What's the number 10. What?
    I currently have 10.2.30.0 and have not complaints except for the fact that around the 10.2.26 update I lost the ability to control saturation, sharpness and contrast.
    Reply
  • Andrei Frumusanu - Saturday, September 14, 2019 - link

    10.28 Reply
  • Redmyth79 - Friday, September 13, 2019 - link

    So you understand what I'm calling bullshit on is the performance charts with GPU and CPU test with the many test. The one thing the Xiaomi Mi 9 truly excels at is CPU, GPU and overall speed and gaming experience always running at the top of any flagship out. The only one I've seen close is the OnePlus 7 pro.
    As far as your end review, I saw your camera info, you are wrong in a few areas here as well.
    The Xiaomi Mi 9 does have a advanced EIS that is not like others but is actually a advanced form of OIS with a extra sensor that watches the movement of the OIS to give it more accurate and faster info to make proper adjustments to make the Mi 9 one of the most stable of all current smartphones in 4k@30 video.
    Any YouTube reviews and also dxomark will show this without question up amongst all the rest.
    If I can post the link to the Mi 9's EIS I will do it here for those to read up on it.
    Last, I don't have a issue with low light pics and most reviews show the same bit it's not of the level of the Note 10+, P30 pro or Pixel without Gcam for night shots.
    gizchina.com/2019/02/17/mi-9-closed-loop-motor-explained/
    Of course you have to put in the http/www part on that link.
    Reply
  • porcupineLTD - Friday, September 13, 2019 - link

    A new fanboy arrives. Reply
  • Redmyth79 - Friday, September 13, 2019 - link

    Fanboy of what I've always been a Galaxy fanboy sense I've owned all from S4-S9+ but the Mi 9 is truly a game changer to me Reply
  • NXTwoThou - Friday, September 13, 2019 - link

    As a fyi, I've taken some rather craptackular shots with my Mi9. I've also taken some astoundingly good ones. Previously I was Lumia 950 and 920 before that. I'd been used to relatively good cameras on my phones. Lack of OIS is a big deal and the EIS doesn't make up for it. I have some health issues that cause trembles, my older phones it made zero difference, with the Mi9 it's very apparent when I'm having a bad day. Even when I'm not having a bad day I can still get bad shots where the same environment had previously been no issue for my previous phone.

    The Mi9 is an insanely good value. I love my phone. I just recognize the issues presented in the article are valid.
    Reply
  • Jon Tseng - Sunday, September 15, 2019 - link

    If you have good benchmarks and data post them rather than ranting.

    Talk is cheap if you're too afraid to back it up.

    Anandtech has years of best in class work around mobile performance analysis. Forgive me if I give them the benefit of the doubt.
    Reply

Log in

Don't have an account? Sign up now