Use ONNX Runtime in a Node app

By Ryze AI Team

This tutorial runs an ONNX model in a Node.js application for image classification.

1. Install

npm install onnxruntime-node sharp. For GPU, use onnxruntime-node-gpu.

2. Load and run

const ort = require('onnxruntime-node');
const session = await ort.InferenceSession.create('model.onnx');
const feeds = { input: new ort.Tensor('float32', data, [1, 3, 224, 224]) };
const results = await session.run(feeds);
console.log(results.output.data);

3. Preprocess images

Use sharp to resize and normalize images to match the model's training preprocessing. Convert to a Float32Array in NCHW layout.

const sharp = require('sharp');
const { data } = await sharp('image.jpg').resize(224, 224).raw().toBuffer({ resolveWithObject: true });
// convert to float32, normalize, reshape to [1,3,224,224]

4. GPU and fallback

Try the GPU package and fall back to CPU on failure. Set the execution provider explicitly when needed.

5. Production tips

Load the session once at startup. Reuse it across requests. Set a concurrency limit, since inference is CPU-bound. Monitor memory usage, as ONNX sessions hold their weights in memory.

Docs: ONNX Runtime JS.