Import your AI Model into Lens Studio to enhance your Lens

SnapML lets you to integrate machine learning models (like the ones created with FieldDay) into your Lens Studio projects. Here's a step-by-step guide on how to export your trained model, and take your Snap Lens to the next level with AI.

Step 1

Exporting your model to Lens Studio

Once you're happy with the performance of your model in FieldDay, you can easily export it using the 'Share' button in FieldDay. Find the 'Lens Studio' option and tap 'Get Template via Email'. In your inbox you'll receive all relevant files, including:

.onnx

Open Neural Network Exchange File (the model)

.lsproj

Lens Studio Template

.js

A script with your labels

Step 2

Importing your model into Lens Studio

To import your model into Lens Studio follow these steps:

  1. Open the Lens Studio template file that you generated,
    or go to Lens Studio and open the FieldDay Classification Template.

  2. With the project open, drag the .onnx file and the Label script file into your project resources.

  3. In the ML component setup pop-up, set the scale and bias input parameters to the following values for all FieldDay models:

Scale

0.0171248

0.0175070

0.0174292

Bias

-2.1179039

-2.0357143

-1.8044444

Step 3

Connect your ML Component

In your Objects panel, select your ML component. This will open the Inspector panel, where you can select your model. Uncheck "Stretch" and select "Screen Crop Texture" as the texture to input the camera feed to your model.

Step 4

Connect your Label Script

In your Objects panel, select the Labels script that lives under the Multi Class Classification object. In its Inspector panel, slide the Labels script you exported from FieldDay or click on the current Label script to replace it with the new one.

Step 5

Connect your Label Script

Adjust the 'Top K' value in the Inspector panel of the "Multi Class Classification" object to a number that is equal to the number of categories you have in your FieldDay model (including the 'Background' category if you have it enabled). The threshold defines the level of confidence your model needs to have to trigger a response. Illustrated below (0.70) is pretty high. Make sure to check the confidence of your model in FieldDay to adjust it. You can see the confidence in the analyse tab, with a double tap on the camera feed inside a fieldDay project.

You're all set! You can use the toggle to display or hide the live analysis view to debug and start adding effects to each label.

© 2023 FieldDay Technologies, Inc.