Skip to content

Instantly share code, notes, and snippets.

View olokobayusuf's full-sized avatar

Yusuf Olokoba olokobayusuf

View GitHub Profile
@olokobayusuf
olokobayusuf / VideoRecorder.cs
Last active December 29, 2023 00:10
An example illustrating recording video and microphone audio in Unity with NatCorder API and NatDevice API.
class VideoRecorder {
MP4Recorder recorder; // Recorder that will record an MP4
CameraInput cameraInput; // Recorder input for recording video from a game camera
AudioDevice audioDevice; // Microphone for recording user audio
void StartRecording () {
// Get a microphone
var query = new MediaDeviceQuery(MediaDeviceCriteria.AudioDevice);
audioDevice = query.current as AudioDevice;
@olokobayusuf
olokobayusuf / GIFCreator.cs
Created November 20, 2023 13:42
Creating an animated GIF image from an array of textures in Unity with VideoKit.
public static async Task<MediaAsset> CreateGIF (Texture2D[] textures) {
// Make sure `textures` is not empty
// Make sure every texture has the same size (width and height)
// Make sure every texture has the `TextureFormat.RGBA32` format
// Create recorder
var width = textures[0].width;
var height = textures[0].height;
var frameRate = 5;
var recorder = await MediaRecorder.Create(
MediaFormat.GIF,
@olokobayusuf
olokobayusuf / RenderTextureDetect.cs
Created February 2, 2023 16:49
Performing object detection on the contents of a render texture with the YOLOX model.
void Detect (RenderTexture renderTexture) {
// Create the YOLOX model
var model = await MLEdgeModel.Create("@natsuite/yolox");
// Create the YOLOX predictor
var predictor = new YOLOXPredictor(model);
// Readback the render texture
AsyncGPUReadback.Request(renderTexture, request => {
// Create an image feature
var feature = new MLImageFeature(request.GetData<byte>(), request.width, request.height);
// Predict
@olokobayusuf
olokobayusuf / MatOutput.cs
Created April 17, 2022 23:06
Integrating NatDevice and OpenCV for Unity with a camera device output that streams camera images into an OpenCV matrix.
/*
* NatDevice
* Copyright (c) 2022 NatML Inc. All Rights Reserved.
*/
namespace NatSuite.Devices.Outputs {
using System;
using OpenCVForUnity.CoreModule;
using OpenCVForUnity.UtilsModule;
@olokobayusuf
olokobayusuf / MLPredict.cs
Last active September 24, 2021 20:00
Creating a predictor for making predictions with ML models.
// Create a classification predictor
var labels = new [] { "cat", "dog", ... };
var predictor = new MobileNetv2Predictor(model, labels);
@olokobayusuf
olokobayusuf / HelloML.cs
Last active September 21, 2021 23:49
The core API in NatML.
// Load the ML model data
var modelData = MLModelData.FromFile("/path/to/model.onnx");
// Deserialize the model
var model = modelData.Deserialize();
// Print a model metadata value
Debug.Log("The model was created by: " + model["Author"]);
// Check the expected input size
var inputType = model.inputs[0] as MLImageType;
Debug.Log($"Model expects image with size: {inputType.width}x{inputType.height}");
@olokobayusuf
olokobayusuf / NatCam+OpenCV.cs
Created October 26, 2018 17:38
A script illustrating the recommended NatCam-OpenCV workflow.
byte[] pixelBuffer;
Mat previewMatrix;
void Start () {
// Start the camera preview
NatCam.Play(DeviceCamera.RearCamera);
// Register for preview events
NatCam.OnStart += OnStart;
NatCam.OnFrame += OnFrame;
}
@olokobayusuf
olokobayusuf / MLModules.cs
Created April 24, 2021 20:22
Modules which simplify model usage in NatML.
// Create a classifier module
var labels = new [] { "goldfish", "tiger shark", ... };
var classifier = new MLClassifier("/path/to/classifier.onnx", labels);
// Classify an image
Texture2D image = ...;
var (label, confidence) = classifier.Predict(image);
Debug.Log($"Image contains {label} with confidence: {confidence}");
@olokobayusuf
olokobayusuf / MLPredict.cs
Last active September 12, 2021 08:05
Make predictions with a NatML model.
// If you have some feature
Texture2D image = ...;
// You can predict easily
var (label, confidence) = predictor.Predict(image);
Debug.Log($"Image contains {label} with confidence {confidence}");
@olokobayusuf
olokobayusuf / ThreadedAsyncTextureInput.cs
Last active April 21, 2021 13:13
Texture input that uses asynchronous readbacks, but commits frames on a background thread.
using NatSuite.Recorders.Inputs;
/// <summary>
/// Recorder input for recording video frames from textures.
/// </summary>
public class ThreadedAsyncTextureInput : ITextureInput {
#region --Client API--
/// <summary>
/// Create a texture input which performs asynchronous readbacks.