SHADERERA®Video made in code · v0.1.2PluginsAgentsGitHub ↗
← PluginsAnalysis & Trackingcomponent · v0.1.0 · MIT
review-av-timelineOfficial · by Shaderera

Sound and motion review

For a reviewer who cannot hear or watch: the film's sound as a spectrogram and waveform with its onsets, how much the picture changes every frame on the same timeline, frame strips around chosen moments, and the measured offset between picture and sound.

Add it to a filmOr ask your agent
“Use the shaderera-plugins skill to add review-av-timeline to this film and wire it in, then run check and show me a still.”
Type
component · Analysis & Tracking
Version
0.1.0 · published
Licence
MIT
Built on
No other plugins
npm
Tags
review, audio sync, lip sync, beat sync, spectrogram, waveform, motion, pacing, frames, ffmpeg, listen, hear, watch, playback, speed, density, timing, on beat, check sync
Source on GitHub ↗
READMEreview-av-timeline

How to
use it.

See the sound and the motion of a rendered film: spectrogram, waveform, onsets and picture change on one timeline; every frame around a moment; the picture-to-sound offset in ms.

README.mdexample
npx shaderera render --out out/film.mp4node tools/av-review.ts out/film.mp4 --at 3.29,8.1,12.45   # the moments that should hit# t 3.290: sound onset 3.286 (-4 ms), picture peak 3.300 (+10 ms), picture − sound +14 ms (0.8 frames)

Then look at out/av/timeline.png and each out/av/at-<t>.png:

  • Sync: the red line is the moment; yellow ticks are where the sound hits. The frame where the picture changes should sit on a tick. Picture up to about 45 ms after the sound reads as in sync; picture before it is noticed sooner, at about 20 ms.
  • Pacing and density: the lower curve is how much each frame differs from the last. Spikes are cuts and hits; a busy band is fast motion; flat stretches are holds. Compare its shape with the spectrogram's sections.

An agent cannot hear or watch playback. This turns both into pictures and numbers it can check, and the person still listens to the final. Needs ffmpeg and ffprobe on PATH, and @napi-rs/canvas (installed with it).