Wireface 2D

Wireface 2D is for pictures. Give it a photo, a painting, anime, pixel art or a lump of clay with a face in it, and the picture itself blinks, breathes, pulls expressions and talks, with lip sync, on a plain canvas. The animation is the one the 3D face and Wireface Desktop use, so the method names are the same and the same code can drive either. To see it first, open Try Wireface 2D; the product page is Wireface 2D.

Get the SDK#

Wireface 2D is part of the Wireface SDK, as plain ES modules on WebGL2 with no runtime dependencies. Download it from your account and follow Get the SDK to add it to a project. The entry points used on this page:

Play a character#

The smallest page is a canvas, a face and a character. Size the canvas with CSS; the face fills it.

import { createFace2d } from '@wireface/sdk/2d';

const face = createFace2d(document.querySelector('canvas'));
await face.useCharacter('sora');   // one of the bundled characters, by id
face.express('happy', 3);          // a smile for three seconds

createFace2d(canvas, options) takes config (any of the settings), assets (the folder the bundled characters and voices are served from) and follow (the head turns a little towards the pointer, on by default). useCharacter(character, { transition }) burns the old character away and the new one in over half a second; pass transition: 0 to switch at once. It accepts a bundled character's id or a character you have made or loaded, and resolves when it is on. Call face.destroy() when you are done with the canvas, to free the GPU.

Make a character from a picture#

A character is a picture plus the data that lets its face move: a 468-point rig fitted over the face, and the preparation that carries that movement onto the picture. createCharacter makes one on the device, with nothing uploaded:

import { createFace2d } from '@wireface/sdk/2d';
import { createCharacter } from '@wireface/sdk/create';

const picture = await createCharacter(file, {   // a File, a Blob or a URL
  kind: '2d',                                   // the picture itself moves
  name: 'Ada',
  artStyle: 'photo',                            // the default; see below
});
const face = createFace2d(canvas);
await face.useCharacter(picture);

MediaPipe finds the face in the browser, then the SDK fits the rig to it, seals the lips and pins the outline. A clear picture with one face works best, straight on, with the mouth closed. Character creation runs offline, needs no account or key, and the MediaPipe files travel in the SDK's assets/mediapipe folder. Other options:

• artStyle tells the face what it is looking at: photo, render3d, anime, cartoon, comic, painting, watercolor, sketch, pixel, clay, vector, creature, robot or other. Settings left on auto follow it: flat or lit shading, how far the head turns in 3D, crisp pixels or smoothing, and which teeth are drawn.
• landmarks is 468 known [x, y] points (0 to 1 of the picture), to skip detection.
• teeth, signal (to cancel) and onProgress are passed to the hosted service when it is used.
• mediapipe is where MediaPipe loads from, if you host its files yourself (see Bundlers).

Drawn and stylised faces: the detection API

MediaPipe reads photos and realistic paintings and 3D renders. It cannot find most drawn, anime, pixel-art or creature faces, and in that case createCharacter throws a WirefaceError with the code face_not_found. For those pictures there is an optional detection API, free with an API key from your account. Pass a rigging client and a 2D picture MediaPipe cannot read goes to it automatically; the finished character comes back to you and Wireface does not keep it. Your permanent key stays on your server and the browser gets short-lived tokens. The client, token endpoint, limits and error codes are all in the detection API section of the developer guide.

import { createRiggingClient } from '@wireface/sdk/rigging';

const rigging = createRiggingClient({ getToken });   // your endpoint returns a short-lived token
const drawing = await createCharacter(file, { kind: '2d', artStyle: 'anime', rigging });

detector chooses where the work happens: 'auto' (the default) tries the device first, 'local' never leaves the device, and 'hosted' always uses the service (it needs rigging).

Save and load characters#

A finished character is yours to keep. exportCharacter returns a .wireface.json file as a Blob, holding the picture, the fitted rig and the settings it needs. It holds no credentials and no code. Loading it back never runs detection and never contacts Wireface, so saved characters play offline.

import { createFace2d, exportCharacter, loadCharacter } from '@wireface/sdk/2d';

const blob = await exportCharacter(picture);          // offer it as ada.wireface.json
const link = Object.assign(document.createElement('a'),
  { href: URL.createObjectURL(blob), download: 'ada.wireface.json' });
link.click();

const again = await loadCharacter('/characters/ada.wireface.json');   // a URL, Blob or ArrayBuffer
await createFace2d(canvas).useCharacter(again);

A saved 2D character plays in createFace2d and in a FaceScreen with engine: '2d', which puts it on a surface in a three.js scene (see Wireface for three.js). useCharacter('some-id') with a string looks the id up in the bundled characters only; to play a file by URL, load it with loadCharacter first.

Expressions, speech and lip sync#

These are the same calls as the 3D face's.

face.express('surprised', 2);       // an expression for 2 seconds (default 4)
face.mood('content', 0.4);          // a standing mood, worn when nothing else is showing
face.mood(null);                    // ...and put it down
face.state('thinking');             // idle, thinking, busy, error, muted or asleep

const audio = new AudioContext();   // make it in a click handler, or the browser keeps it silent
const clip = await face.clip('sora');
await face.say(clip, { audioContext: audio, onCaption: text => (caption.textContent = text) });
face.hush();                        // stop mid-line

• Expressions. The moods are neutral, happy, excited, amused, surprised, curious, thinking, concerned, sad, afraid, frustrated, calm, content, bored and glum. Everyday words such as angry, worried and laugh map onto them, and an unknown name throws. Make your own with defineExpression('smirk', { smileL: 0.8, browUpR: 0.5 }), using the channels in CHANNELS.
• Recorded lines. face.clip(id) loads a line from the voices folder: sound and a mouth track aligned phoneme by phoneme, so the lips close on every m, b and p. Each bundled character has one. Without an audioContext, say mimes the line silently. The promise resolves when the line ends or is stopped.
• Any audio, live. face.listenTo(audioContext, source, { gain }) reads the mouth from sound as it plays: a microphone stream, an <audio> element or an audio node. It returns a function that stops it. mouth_gain and sync_offset_ms in the settings tune it.
• Streamed speech. StreamVoice plays speech that arrives as raw PCM, such as a text-to-speech or Gemini Live stream, and works the mouth ahead of each piece. Conversation from @wireface/sdk/2d/live wraps a whole Gemini Live chat around a face: it needs a token endpoint on your server, which is where your Gemini key stays.
• Appearing. await face.disappear() burns the picture away from the bottom up, and await face.appear() brings it back.

face.on(fn) reports speaking and caption events, and returns an unsubscribe function. face.project([u, v]) turns a point on the picture (0 to 1) into canvas pixels, and face.unproject([x, y]) goes back, for placing a speech bubble or a hat. Your own sound, from any text-to-speech service, is lip-synced locally with listenTo.

Bundled characters#

Thirteen characters, one for each of the art styles most pictures fall into, each with a recorded line. They live in the SDK's assets/2d/characters folder. await face.catalogue() lists them with their name, style, voice, persona and greeting, and the URL of each one's picture and .wireface.json.

Settings#

Pass settings to createFace2d as config, or change them at any time with face.set({ ... }), which returns the whole config and throws a ConfigError naming anything unknown or out of range. face.config shows how they stand now.

Settings also fit in a file. validateConfig checks one, configDiff returns only what differs from the defaults, and the package's face2d.schema.json gives editors such as VS Code autocomplete when a config file names it as its $schema. The SDK's examples/2d/config/ folder has samples.

Bundlers and hosting#

With no build step, serve the SDK's dist/ and assets/ folders together and import from dist/2d.js by path; the face finds the characters and voices in assets/2d/ beside the bundles. With a bundler, which moves modules around, tell the face where the assets are:

1. Copy node_modules/@wireface/sdk/assets into your public folder, for example public/wireface/.
2. Pass the 2D folder: createFace2d(canvas, { assets: '/wireface/2d/' }).
3. For createCharacter, point MediaPipe at its files:

const picture = await createCharacter(file, {
  kind: '2d',
  mediapipe: {
    bundle: '/wireface/mediapipe/vision_bundle.mjs',
    wasm: '/wireface/mediapipe/wasm',
    model: '/wireface/mediapipe/face_landmarker.task',
  },
});

Serve .wasm files as application/wasm. If your bundler pre-bundles dependencies (Vite does), exclude @wireface/sdk from that, so it can find its own files. The microphone worklet and teeth pictures stay with the bundle.

Where next#

• Wireface SDK developer guide: the detection API, export and offline playback in full.
• Wireface for three.js: put a 2D character on a screen, a can or a painting in a 3D scene.
• Embedding: a talking face in a single iframe, with no code.
• Try Wireface 2D in the browser, or read about the product.

Adding Wireface to an app, game or website is free under the Personal Licence. If you charge for what you build, see Licensing for the licence you need.