# i.MX8MP Vision Processing
**URL:** https://community.solid-run.com/t/i-mx8mp-vision-processing/263
**Category:** SolidRun DevCuts
**Tags:** AI, basler-camera, mx8mp
**Created:** [March 14, 2022, 2:54pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263 "2022-03-14T14:54:52Z")
**Posts on this page:** 14
**Page:** 1
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [March 14, 2022, 2:54pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/1 "2022-03-14T14:54:52Z")
In this weeks video we cover the basics for the Basler camera integration into the BSP and how the imx8-isp is integrated into the video. Then we spend the rest of the video talking about how I tuned a generic python opencv script to make use of the onboard NN accelerator and tuned the pipeline process in order to be able to process 1080p at 30 frames per second.
Links referenced in the video:
NXP Documentation
- [i.MX 8M Plus | Cortex-A53/M7 | NXP Semiconductors](https://www.nxp.com/products/processors-and-microcontrollers/arm-processors/i-mx-applications-processors/i-mx-8-processors/i-mx-8m-plus-arm-cortex-a53-machine-learning-vision-multimedia-and-industrial-iot:IMX8MPLUS?tab=Documentation_Tab)
- [https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf](https://www.nxp.com/docs/en/user-guide/IMX-MACHINE-LEARNING-UG.pdf)
Python Script Gist and Web Page to download the tflite model used.
- [Quick script to get full performance using the NN accelerator on the iMX8MP · GitHub](https://gist.github.com/jnettlet/9ec0732fec4703a58d25ad4cf26a2491)
- [Detección de objetos | TensorFlow Lite](https://www.tensorflow.org/lite/examples/object_detection/overview)
If you have any additional questions or comments feel free to drop them below.
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 15, 2022, 7:25pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/2 "2022-05-15T19:25:28Z")
Hey @jnettlet
I am using the zeus imx-5.4.47-2.2.0.xml manifest
I’v added the following to my local.conf
IMAGE\_INSTALL\_append += " packagegroup-imx-ml"
TOOLCHAIN\_TARGET\_TASK\_append += “tensorflow-lite-staticdev tensorflow-lite-dev armnn-dev onnxruntime-dev”
now i can’t seem to find vx\_delegate.so in /usr/lib i reckon that i need to compile tensorflow-lite with -DLITE\_XNNPACK and replace its revision with TM\_VX or something?
please help me out understanding how i can communicate with the NPU
btw does it work without a special delegator?
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 16, 2022, 6:03am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/3 "2022-05-16T06:03:54Z")
Thanks for reaching out. Is it possible for you to use the hardknott branch? The NXP BSP for the machine learning features is much better in that release. That is the version I have used for all of my testing and videos.
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 16, 2022, 6:42am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/4 "2022-05-16T06:42:24Z")
Hey Jnettlet
I’m afraid this is not possible for us, we need to figure out how to launch it in zeus 5.4
i can maybe use other delegator such as nnapi or armnn but i can’t seem to understand how to add those so i can have a delegate shared object.
Thanks for the quick response
daniel
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 16, 2022, 7:40am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/5 "2022-05-16T07:40:32Z")
It looks like Zeus is using the older tflite integration that is not using OpenVX and delegates. I recommend you read NXP’s documentation for doing machine learning with Zeus. This is already a 2+ year old BSP and a lot has changed from the original software that was released for the iMX8MP
> **[i.MX\_Machine\_Learning\_User's\_Guide.pdf](https://community.nxp.com/pwmxy87654/attachments/pwmxy87654/imx-processors/166787/1/i.MX_Machine_Learning_User's_Guide.pdf)**
>
> 2.43 MB
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 16, 2022, 9:30am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/6 "2022-05-16T09:30:02Z")
I’ve managed to run an example of the tensorflow-lite which uses nnapi, i managed to see the NPU in action according to the benchmark. but i noticed fallback to the cpu all of the times, and very high warming times. so i am willing to make a hardknott yocto for the sake of getting better results.
if you can please tell me how did you configurate your local.conf/bblayers.conf on hardknott to use openvx delegator with tensorflow lite?
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 16, 2022, 9:41am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/7 "2022-05-16T09:41:12Z")
You should just need to add `IMAGE_INSTALL_append += " packagegroup-imx-ml"` as you have done above. For the iMX8M that will pull in `tensorflow-lite-vx-delegate`
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 16, 2022, 9:44am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/8 "2022-05-16T09:44:33Z")
Thanks Jnettlet i will try asap
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 17, 2022, 12:35am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/9 "2022-05-17T00:35:20Z")
i am trying to follow your advise about hardknott but i am struggling to boot my som after i uuu the hardknott img i’ve created following this guide:[GitHub - SolidRun/meta-solidrun-arm-imx8](https://github.com/SolidRun/meta-solidrun-arm-imx8/tree/hardknott-imx8mp)
i am building imx-image-core and i’ve added image\_install\_append=packagegroup-imx-ml and parallel make and num of threads, that’s it.
i flush the som using uuu with the img and the evk.
but w/e i boot it i get this troublesome output, i’ve atttached an image so you can see what i’m refereing to
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 17, 2022, 3:58am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/10 "2022-05-17T03:58:18Z")
It looks like u-boot is incorrectly selecting the root device that is passed to extlinux.conf…or the extlinux.conf file is incorrectly being generated.
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 17, 2022, 12:58pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/11 "2022-05-17T12:58:17Z")
my generated extlinux.conf:
# Generic Distro Configuration file generated by OpenEmbedded
LABEL NXP i.MX Release Distro
KERNEL ../Image
FDTDIR ../
APPEND root=/dev/mmcblk1p2 rootwait rw quiet console=${console} ${bootargs} rootwait rw console=${console},${baudrate}
please help
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 17, 2022, 1:03pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/12 "2022-05-17T13:03:41Z")
okay that is the issue there. It should be /dev/mmcblk2p1. I will have to see why NXP’s bitbake recipe is hard-coding that rather than letting u-boot determine that based on the distro-config.
---
### Author:  [@daniel.e](https://community.solid-run.com/u/daniel.e)
#### Post date: [May 17, 2022, 4:14pm UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/14 "2022-05-17T16:14:35Z")
Update:
so i succesfully burnt hardknott and booted it up correctly, i tested the vx\_delegator but i can’t seem to understand why the benchmark process from nxp is taking a single core by 100%. isn’t the npu should remove the overhead from the cpu? i am expecting to see 0 cpu usage during interaction with the NPU. am i wrong?
---
### Author:  [@jnettlet](https://community.solid-run.com/u/jnettlet)
#### Post date: [May 18, 2022, 4:05am UTC](https://community.solid-run.com/t/i-mx8mp-vision-processing/263/15 "2022-05-18T04:05:45Z")
There is processing that is happening besides just running the model against the data on the NPU. There is image processing, and other pipeline processing that is happening. The only thing that is offloaded to the NPU is the inferencing.