Machine Learning Inference Performance

AIMark 3

AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.

鲁大师 / Master Lu - AIMark 3 - InceptionV3 鲁大师 / Master Lu - AIMark 3 - ResNet34 鲁大师 / Master Lu - AIMark 3 - MobileNet-SSD 鲁大师 / Master Lu - AIMark 3 - DeepLabV3

In AIMark, the Oppo Reno 10x ends up quite adequately amongst the Snapdragon 855 devices who come shipping with the corresponding Qualcomm libraries to make the benchmark work. We do note that other S855 phones have a slight edge here, likely due to their more optimised software stack.

AIBenchmark 3

AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.

We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.

AIBenchmark 3 - NNAPI CPU

We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.

AIBenchmark 3 - 1 - The Life - CPU/FP AIBenchmark 3 - 2 - Zoo - CPU/FP AIBenchmark 3 - 3 - Pioneers - CPU/INT AIBenchmark 3 - 4 - Let's Play - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/FP AIBenchmark 3 - 7 - Ms. Universe - CPU/INT AIBenchmark 3 - 8 - Blur iT! - CPU/FP

In the CPU accelerated workloads, we see that the Oppo Reno 10x absolutely stands out compared to all other Snapdragon devices, having nothing in common with the rest of the pack in terms of the resulting performance figures. The phone ends up with one performance regression, one test that didn’t complete, but otherwise the performance looks to actually be better than that of what’s available on other Snapdragon 855 phones.

The only explanation that I can offer here is that it looks like rather than using Qualcomm’s NN driver stack, the phone is falling back to the default Android Tensorflow libraries. The interesting thing here is indeed that the phone looks to be performing better with these libraries than that the ones that are shipping with the Qualcomm BSP, something the company should certainly look into.

AIBenchmark 3 - NNAPI INT8

AIBenchmark 3 - 1 - The Life - INT8 AIBenchmark 3 - 2 - Zoo - Int8 AIBenchmark 3 - 3 - Pioneers - INT8 AIBenchmark 3 - 5 - Masterpiece - INT8 AIBenchmark 3 - 6 - Cartoons - INT8

AIBenchmark 3 - NNAPI FP16

AIBenchmark 3 - 1 - The Life - FP16 AIBenchmark 3 - 2 - Zoo - FP16 AIBenchmark 3 - 3 - Pioneers - FP16 AIBenchmark 3 - 5 - Masterpiece - FP16 AIBenchmark 3 - 6 - Cartoons - FP16 AIBenchmark 3 - 9 - Berlin Driving - FP16 AIBenchmark 3 - 10 - WESPE-dn - FP16

AIBenchmark 3 - NNAPI FP32

AIBenchmark 3 - 10 - WESPE-dn - FP32

Unfortunately for the rest of the AI Benchmark tests which make use of NNAPI acceleration for INT8, FP16 and FP32 models, the Reno 10x just didn’t have the correct drivers and thus wasn’t able to perform the tests.

If there’s one thing that’s worse than bad performance, it’s not able to perform a task at all. I’m not sure if the Reno 10x performs any differently in a global firmware version of if this issue is solely a characteristic of the Chinese variant. In any case, I find it quite disappointing for the machine learning ecosystem as Oppo is a major vendor, and the Reno is its flagship device. How are application developers supposed to embrace machine learning when we the software situation still has such enormous draw-backs?

System Performance GPU Performance
POST A COMMENT

40 Comments

View All Comments

  • Andrei Frumusanu - Wednesday, September 18, 2019 - link

    Forgot to edit that subtitle. Reply
  • DanNeely - Wednesday, September 18, 2019 - link

    Page1: "The display is an AMOLED panel featuring a 2340 x 1080 resolution and comes at a rather large diameter of 6.6”. " diameter should be diagonal. Reply
  • DanNeely - Wednesday, September 18, 2019 - link

    "Oppo alleviates this concern by including a small raised nub below the cameras, causing the phone to never actually be flush against any surfaces and thus vastly reducing possible scratches of the back glass near the cameras."

    Does this mean that if you set it down on the table and try to use it you're going to have a wobbly bad time?
    Reply
  • Andrei Frumusanu - Wednesday, September 18, 2019 - link

    It wobbles if you press the top of the screen, the bottom 2/3rds are stable. Reply
  • trivik12 - Wednesday, September 18, 2019 - link

    Since almost all flagship phones have night mode, can we do a shoot out and provide us strength and weakness plus overall winner. I guess best time would be when Pixel 4 is released. Reply
  • flyingpants265 - Wednesday, September 18, 2019 - link

    -Needs front stereo speakers which allows you to hear youtube videos, movies, emulator games, camera videos, etc in full volume.
    -And wireless charging which is extremely useful, no port wear, start charging faster/with one hand, magnetic charging mount in the car.. Still use it all the time on my Nexus 5. Wireless charging pad cost me $3.00 on ebay.
    -The missing 3.5 headphone jack is not really ideal..
    Reply
  • melgross - Wednesday, September 18, 2019 - link

    I’m not convinced that a pop up camera is a good idea, particularly if all that mechanics and electronic is there to just eliminate a hole in the screen.

    Notice that there’s no water and dust rating. That’s because it’s almost impossible to keep water and dust out of the slot. I can see grit from wind picking it up, being deposited inside that slot, eventually scratching the lenses and eventually causing the module to grind to a halt. Moisture too. It will remain in the slot for some time, causing problems.
    Reply
  • eastcoast_pete - Wednesday, September 18, 2019 - link

    Thanks Andrei! Agree with you and others here that the pop up camera is a gimmick that adds complexity and weight, not to mention anything that moves is another thing that can break. Too bad Oppo didn't put the extra weight and complexity to make a really good periscope style optical zoom camera - if done correctly, they can be spectacular.
    @Andrei: Lastly, one question and request: what about the call quality of this and other phones? Please add a sentence or two on this to your reviews, good call quality is a non-negotiable for me and probably many others.
    I do use my smartphone as my main phone, and had to bench/retire my Xiaomi Mi Max 3 phone due to really horrible call quality (on both sides, got many complaints over poor voice quality). I loved many aspects of that phone, especially the big screen and battery, but it was basically useless as a daily driver. Now back to an older LG phone, not as good otherwise, but I can make and receive clear phonecalls.
    Reply
  • PreacherEddie - Wednesday, September 18, 2019 - link

    I believe the top of page 2 comes from a previous review and you need to edit it for the Oppo. Reply
  • edsib1 - Wednesday, September 18, 2019 - link

    I have no idea how your Oppo10x zooms scores so low in benchmarks. My UK version scores very high in all benchmark scores. Antutu score of 370k, geekbench 4score 3514/11314 . Pcmark work2 performance score is 10564 Reply

Log in

Don't have an account? Sign up now