On-device AI is becoming increasingly popular, and some time ago, we began exploring this field. Given our passion for React Native, we decided to merge these interests, resulting in React Native ExecuTorch.

What is ExecuTorch?

ExecuTorch is a novel AI framework from the PyTorch team that provides tools to export and run PyTorch models on edge devices like mobile phones and microcontrollers. It enables exporting models and running them locally without requiring API calls. If you want to learn more about ExecuTorch check their docs and our previous blogs about it here, here, and here.

Why React Native library?

Recently, we took on the task of developing a photo eraser application. Throughout the process, we realized that many functionalities written in native code could be abstracted into public API in JavaScript. That’s how we came up with the idea of React Native ExecuTorch.

Our primary goal is to ensure seamless integration between React Native and ExecuTorch, allowing developers without extensive AI expertise to run models in their applications with minimal overhead. Additionally, on-device models enable them to create AI-based applications that prioritize user privacy, as there is no need for the transfer of confidential data. This approach also saves significantly on infrastructure costs because developers are not required to host models or maintain additional backends.

How does it work?

In the v0.1.0 version, our library allows you to run large language models such as LLaMA 3.2 directly on mobile devices.

Public API consists of one hook called useLLM

const llama = useLLM({
  modelSource: 'https://.../llama3_2.pte',
  tokenizer: require('../assets/tokenizer.bin'),
});

const message = 'Hi, who are you?';
await llama.generate(message);

return (
    <Text>{llama.response}</Text>
)

This brief code snippet demonstrates how to load a model along with its tokenizer and generate a response from it. You can provide any retrievable resource as the ‘modelSource’ and ‘tokenizer’, the library will take care of the rest. To get more details about loading models and working with React Native ExecuTorch refer to our documentation.

Supported Models

With the initial release, we are offering support for small llama models which can be run on most of the phones. We currently support:

• LLaMA 3.2 1B(Quantized — QLoRA or Spin-Quant)
• LLaMA 3.2 3B(Quantized — QLoRA or Spin-Quant)
• LLaMA 3.2 1B
• LLaMA 3.2 3B

For your convenience, it’s recommended to use quantized models, as they are more computationally efficient and can run on less powerful devices. However, you should be aware that devices without at least 8 GBs of RAM running those models could lead to application crashes.

Where can I find exported models?

As you already know, React Native ExecuTorch allows you to run models directly on-device, which means that you will need to provide exported models for each of your users. Given the large size of the models, they cannot be included in the React Native bundle. To simplify this process, we’ve wanted to find a practical solution that enhances developer experience. We’ve decided to host exported models on a hugging face repository allowing you to download them directly as needed. Thanks to that you can always be sure that the versions of models and library are compatible. Direct URLs to those files are distributed within our project and you can check them out here.

Alternatively, you can export models yourself — our docker image makes this straightforward — and distribute them to your users in the most suitable way. To learn more about obtaining models check out the dedicated section in our docs.

Plans for the future

When we first talked about React Native ExecuTorch we set a goal to provide tools for React Native developers to export and run their favorite model within their applications. Our team understood that it would be a big challenge and it would take a lot of time and effort. Today, we are proud to share the first steps of our journey, offering APIs for running large language models.

In the next release, you can expect support for more language models — maybe some bigger ones- as well as computer vision models such as SAM which we are using in our photo eraser application. We are also actively working on audio processing so hopefully, we will be able to share something about it soon. Follow us on our social media and stay tuned for more updates on React Native ExecuTorch.

Try it out!

React Native ExecuTorch is available on npm. We are counting on you and would like to know what you think so make sure to start discussions and report any issues via GitHub.

This project has been built and is maintained by Software Mansion. We’re a software agency with experience in building web and mobile apps. We are React Native Core Contributors, creators of many libraries loved by the community including React Native Reanimated and Gesture Handler.

We can help you build your next dream product — hire us.