AI features

npm install @video-ultra/ai

Background removal (on the device)

import { createBackgroundRemover } from '@video-ultra/ai';

<VideoEditor backgroundRemover={createBackgroundRemover()} />;

Removes the background of photos and image stickers on the device (a small model on phones, a sharper one on computers); nothing is uploaded and it can be undone.

AI editing (through your server)

import { createChatAssistant } from '@video-ultra/ai';

<VideoEditor assistant={createChatAssistant({ endpoint: '/api/assistant' })} />;

An Assistant panel where people type what they want — "make it warmer", "add a title at the start", "a fade between every clip". Your endpoint forwards to any OpenAI-compatible chat model; a description of the project (no media) is sent, and the answer is applied as one undo step. Keep the API key on your server.

Text to speech (through your server)

import { createSpeechEndpoint } from '@video-ultra/ai';

<VideoEditor speech={createSpeechEndpoint({ endpoint: '/api/speech' })} />;

"Text to speech" in the Audio panel and "Read aloud" on texts add a spoken sound clip, from any OpenAI-compatible /audio/speech endpoint.

The Speed slider works with every voice service: when yours ignores the speed, pass appliesSpeed: false (or leave appliesSpeed out of your own SpeechGenerator) and the editor asks for normal speed, then plays the clip at the chosen speed with the voice's pitch kept.