Software Architecture of Qualcomm AI engine SDK

         Software Architecture of AI engine SDK




The Software architecture of Qualcomm Ai engine Direct API and its associated software stack , which facilitates the construction , optimization and execution of network models on the hardware accelerator cores. 

Key components of Software stack:-

1. Device:- Its virtual representation of the hardware where all the action happens. It helps  manages
and organize the hardware resources(like cores) needed to run your AI programs.

2. Backend:- Its the main hub that oversees everything and manages all the tools needed to run your AI programs effectively. It keeps tracks of all the operations you can do and make sure everything runs smoothly.

3. Context :- It can be visualize as where your AI program lives and breathes. It holds AI networks (the brain of AI model)and helps them to talk each other and share information.

4. Graph :- It is kind of blueprint of your AI program. It's like a map that shows how everything is connected and what each part does. Its made up of different parts(nodes) that perform specific tasks and connect together to make your program work.

5. Operation package registry:- It is like a library that holds all the tools you can use in your AI program. It's a big list of different things you can do . You can choose from the things that already available(built in operations) or bring your own tools(custom operations) to make our model work even better.

In simple terms , we can think of the device as the place where your AI model runs, backend as the manager  that keeps everything organized, the context as the environment where your AI model is present, the graph as the blueprint of the program and the operation package registry as the library of tools you can use to build your program. Together, they form the foundation for creating and running AI applications efficiently on device.

Comments

Post a Comment

Popular posts from this blog

T02:- How to setup the Qualcomm Neural Processing SDK

Image
    Setup of Qualcomm Neural Processing SDK We will now understand how we can setup the Qualcomm Neural Processing Engine SDK in our system   -> To start  with first thing we have to understand what is required for this setup.       1 . Oracle Virtual Box  https://www.virtualbox.org/wiki/Downloads       2 . Ubuntu 20.04   https://releases.ubuntu.com/focal/      We can also work with WSL2 environment that will work on Windows 10/11.       3 . Any Machine Learning Framework         3.1 Tensorflow         3.2 Pytorch         3.3 Tensorflow lite         3.4 Onnx       4 . Android NDK       5 . Microsoft Visual C++(Resdistrubution) So now we will go to the practical aspect of the setup of the Neural Processing SDK(SNPE). 1 .We will first download the Virtual box as per the requirement. ...
Read more

T01:- Introduction to Ondevice AI

Image
                Understanding Of On-device Ai Smart stuff like phones and cars are getting smarter . They can crunch data and learn without needing the internet all the time. Qualcomm is a big part of making this happens.                                       Fig - 1.1(On-device AI) So, What is On-device AI? On-device means smart devices like phones or smartwatches can work without need of constant internet connection. Its like having a live brain inside your gadgets, making quick decisions and performing task without the need of live servers. This technology is good for things like voice assistants and image recognition , as it reduces the latency and ensures privacy by keeping data local.                                           Fig- 1.2 ( ON-...
Read more

Integration Workflow of Ai engine sdk

Image
                        Integration Workflow . Training and Inference workflow:- This diagram illustrates the difference between training a neural       network(done on server) and running inference(making predictions) on device. Training involves creating and optimizing a neural network model using large datasets, typically done off device. Inference is the process of using the trained model to make predictions or decisions based on new data which is usually done on device where the application runs. .Qualcomm AI engine Direct integration workflow:-  It details how to integrate trained deep learning(DL networks) into applications using Qualcomm AI engine SDK. It involves converting the trained model into format suitable  for execution on the Qualcomm chipsets and integrating them into applications. Now, lets connect these workflows with an explanation of the integration process. Integration WorkST...
Read more