Debugging compositions

Overview

Debugging video rendering can be challenging because issues may only appear during headless rendering or at specific frames. Helios provides multiple debugging tools to help diagnose problems.

Diagnostics API

The Helios.diagnose() static method checks environment capabilities.

Client-side diagnostics

import { Helios } from '@helios-project/core';

const report = await Helios.diagnose();

console.log(report);
// {
//   waapi: true,
//   webCodecs: true,
//   offscreenCanvas: true,
//   webgl: true,
//   webgl2: true,
//   webAudio: true,
//   colorGamut: 'p3',
//   videoCodecs: { h264: true, vp8: true, vp9: true, av1: false },
//   audioCodecs: { aac: true, opus: true },
//   videoDecoders: { h264: true, vp8: true, vp9: true, av1: false },
//   audioDecoders: { aac: true, opus: true },
//   userAgent: 'Mozilla/5.0...'
// }

Diagnostic fields

waapi - Web Animations API support (required for CSS animations) webCodecs - WebCodecs API support (required for canvas rendering) offscreenCanvas - OffscreenCanvas support (improves performance) webgl / webgl2 - WebGL support (required for Three.js, Pixi.js) webAudio - Web Audio API support (required for audio mixing) colorGamut - Display color gamut ('srgb', 'p3', 'rec2020') videoCodecs - Supported video encoders (H.264, VP8, VP9, AV1) audioCodecs - Supported audio encoders (AAC, Opus) videoDecoders - Supported video decoders audioDecoders - Supported audio decoders

Renderer diagnostics

For server-side rendering diagnostics:

import { Renderer } from '@helios-project/renderer';

const renderer = new Renderer({ mode: 'canvas' });
const diagnostics = await renderer.diagnose();

console.log(diagnostics);
// {
//   browser: {
//     webCodecs: true,
//     codecs: {
//       h264: { supported: true, hardware: true, alpha: false },
//       vp8: { supported: true, hardware: false, alpha: false },
//       vp9: { supported: true, hardware: false, alpha: false },
//       av1: { supported: false, hardware: false, alpha: false }
//     }
//   },
//   ffmpeg: {
//     version: '4.4.2',
//     encoders: ['libx264', 'libx265', 'libvpx', ...],
//     hwaccels: ['videotoolbox', 'cuda'],
//     filters: ['scale', 'overlay', ...]
//   }
// }

Common issues

WebCodecs not supported

if (!report.webCodecs) {
  console.warn('WebCodecs not available. Use DOM rendering mode.');
  const renderer = new Renderer({ mode: 'dom' });
}

No hardware acceleration

const diagnostics = await renderer.diagnose();
const hasHardware = Object.values(diagnostics.browser.codecs)
  .some(codec => codec.hardware);

if (!hasHardware) {
  console.warn('No hardware encoders available. Rendering will be slower.');
}

WebGL not available

if (!report.webgl && !report.webgl2) {
  console.error('WebGL not supported. Three.js/Pixi.js will not work.');
}

Headed mode

Run rendering in a visible browser window to see what’s happening.

Using CLI

helios render composition.html --no-headless

This opens a browser window and shows the rendering process in real-time.

Using API

import { Renderer } from '@helios-project/renderer';

const renderer = new Renderer({
  mode: 'canvas',
  browserConfig: {
    headless: false  // Show browser window
  }
});

await renderer.render('composition.html', 'output.mp4');

Headed mode is slower than headless because it must render to the screen. Use only for debugging.

Remote debugging

Connect Chrome DevTools to the headless browser for live debugging.

Enable remote debugging

import { Renderer } from '@helios-project/renderer';

const renderer = new Renderer({
  mode: 'canvas',
  browserConfig: {
    headless: true,
    args: [
      '--remote-debugging-port=9222'  // Enable DevTools Protocol
    ]
  }
});

Connect DevTools

Start the render
Open Chrome and navigate to chrome://inspect
Click “Configure” and add localhost:9222
Click “inspect” under “Remote Target”
Use Console, Network, Performance tabs as normal

Debugging tips

Inspect DOM state

// In DevTools console
document.getAnimations().forEach(a => {
  console.log(a.id, a.currentTime, a.playState);
});

Check canvas output

const canvas = document.querySelector('canvas');
canvas.toBlob(blob => {
  const url = URL.createObjectURL(blob);
  console.log('Canvas frame:', url);
});

Monitor memory

if (performance.memory) {
  console.log('Heap:', performance.memory.usedJSHeapSize / 1024 / 1024, 'MB');
}

Playwright trace viewer

Playwright can record a complete trace of the rendering session.

Enable trace recording

import { Renderer } from '@helios-project/renderer';

const renderer = new Renderer({ mode: 'dom' });

await renderer.render('composition.html', 'output.mp4', {
  tracePath: 'trace.zip'  // Save trace
});

View the trace

npx playwright show-trace trace.zip

The trace viewer shows:

Timeline - Frame-by-frame rendering progress
Screenshots - Visual output at each step
Network - Asset loading and requests
Console - Log messages and errors
Source - Code execution with source maps

Debugging with traces

Find slow frames Look for gaps in the timeline where rendering takes longer than expected. Check asset loading Verify fonts, images, and videos load before rendering starts. Inspect console errors Trace viewer captures all console output, including warnings and errors.

Frame-by-frame debugging

Inspect output at specific frames.

Export frames for inspection

import { Helios } from '@helios-project/core';

const helios = new Helios({ duration: 5, fps: 30 });

helios.subscribe((state) => {
  // Render frame
  renderer.render(scene, camera);
  
  // Export specific frames
  if ([0, 30, 60, 90].includes(state.currentFrame)) {
    exportFrame(state.currentFrame);
  }
});

function exportFrame(frame: number) {
  canvas.toBlob((blob) => {
    const url = URL.createObjectURL(blob!);
    console.log(`Frame ${frame}:`, url);
    
    // Download for inspection
    const a = document.createElement('a');
    a.href = url;
    a.download = `frame-${frame}.png`;
    a.click();
  });
}

Seek to specific frame

const helios = new Helios({ duration: 10, fps: 30 });

// Jump to frame 150 (5 seconds at 30fps)
helios.seek(150);

// Wait for assets to stabilize
await helios.waitUntilStable();

// Inspect state
console.log('Frame 150 state:', helios.currentFrame.value);

Console logging

Use console logs strategically to track state.

Log frame state

helios.subscribe((state) => {
  if (state.currentFrame % 30 === 0) {
    console.log('Frame:', state.currentFrame);
    console.log('Time:', state.currentTime);
    console.log('Playing:', state.isPlaying);
  }
});

Log animation state

helios.subscribe(() => {
  const animations = document.getAnimations();
  animations.forEach((anim, i) => {
    console.log(`Animation ${i}:`, {
      currentTime: anim.currentTime,
      playState: anim.playState,
      playbackRate: anim.playbackRate
    });
  });
});

Log driver activity

Create a debug driver wrapper:

import { TimeDriver, DomDriver } from '@helios-project/core';

class DebugDriver implements TimeDriver {
  private inner: TimeDriver;

  constructor(inner: TimeDriver) {
    this.inner = inner;
  }

  init(scope: unknown) {
    console.log('[Driver] init', scope);
    this.inner.init(scope);
  }

  update(timeInMs: number, options: any) {
    console.log('[Driver] update', timeInMs, options);
    this.inner.update(timeInMs, options);
  }

  async waitUntilStable() {
    console.log('[Driver] waitUntilStable');
    await this.inner.waitUntilStable();
    console.log('[Driver] stable');
  }
}

const helios = new Helios({
  duration: 10,
  fps: 30,
  driver: new DebugDriver(new DomDriver())
});

Common rendering issues

Animations don’t sync

Symptom: CSS animations play independently of Helios timeline Solution: Enable autoSyncAnimations

const helios = new Helios({
  duration: 10,
  fps: 30,
  autoSyncAnimations: true  // Automatically pause and sync
});

Fonts not loading

Symptom: Text renders with fallback fonts Solution: Wait for fonts to load

// Wait for fonts
await document.fonts.ready;

// Or use waitUntilStable
await helios.waitUntilStable();

helios.play();

Images appear blank

Symptom: Images don’t render or show as broken Solution: Preload images

const images = document.querySelectorAll('img');
const promises = Array.from(images).map(img => 
  img.complete ? Promise.resolve() : img.decode()
);

await Promise.all(promises);

Canvas shows old frame

Symptom: Canvas doesn’t update or shows stale content Solution: Ensure you’re rendering on every frame

helios.subscribe((state) => {
  // Clear canvas
  ctx.clearRect(0, 0, canvas.width, canvas.height);
  
  // Draw new frame
  drawFrame(state);
});

Audio out of sync

Symptom: Audio doesn’t match video timeline Solution: Use data-helios-offset attribute

<audio 
  data-helios-track-id="bgm"
  data-helios-offset="2.5"
  src="music.mp3"
></audio>

Memory leak during preview

Symptom: Browser slows down or crashes during long previews Solution: Dispose and recreate Helios instance

let helios = new Helios({ duration: 10, fps: 30 });

// After some time
helios.dispose();
helios = new Helios({ duration: 10, fps: 30 });

Performance debugging

Measure frame render time

let totalTime = 0;
let frameCount = 0;

helios.subscribe((state) => {
  const start = performance.now();
  
  // Your render code
  renderer.render(scene, camera);
  
  const duration = performance.now() - start;
  totalTime += duration;
  frameCount++;
  
  if (state.currentFrame === state.duration * state.fps - 1) {
    console.log('Average frame time:', totalTime / frameCount, 'ms');
  }
});

Identify bottlenecks

helios.subscribe((state) => {
  performance.mark('frame-start');
  
  performance.mark('physics-start');
  updatePhysics();
  performance.mark('physics-end');
  
  performance.mark('render-start');
  renderer.render(scene, camera);
  performance.mark('render-end');
  
  performance.measure('physics', 'physics-start', 'physics-end');
  performance.measure('render', 'render-start', 'render-end');
  performance.measure('total', 'frame-start', 'render-end');
  
  if (state.currentFrame % 30 === 0) {
    const entries = performance.getEntriesByType('measure');
    entries.forEach(entry => {
      console.log(entry.name, entry.duration.toFixed(2), 'ms');
    });
    performance.clearMarks();
    performance.clearMeasures();
  }
});

Debug checklist

When debugging a render issue:

✅ Run Helios.diagnose() to check environment
✅ Enable headed mode to see visual output
✅ Check browser console for errors
✅ Verify assets load with waitUntilStable()
✅ Test with simplified composition
✅ Export specific frames for inspection
✅ Use Playwright trace for detailed analysis
✅ Profile with Chrome DevTools Performance tab

Next steps

Learn about custom drivers for specialized workflows
Optimize performance for faster renders
Explore canvas and WebGL rendering patterns

​Overview

​Diagnostics API

​Client-side diagnostics

​Diagnostic fields

​Renderer diagnostics

​Common issues

​Headed mode

​Using CLI

​Using API

​Remote debugging

​Enable remote debugging

​Connect DevTools

​Debugging tips

​Playwright trace viewer

​Enable trace recording

​View the trace

​Debugging with traces

​Frame-by-frame debugging

​Export frames for inspection

​Seek to specific frame

​Console logging

​Log frame state

​Log animation state

​Log driver activity

​Common rendering issues

​Animations don’t sync

​Fonts not loading

​Images appear blank

​Canvas shows old frame

​Audio out of sync

​Memory leak during preview

​Performance debugging

​Measure frame render time

​Identify bottlenecks

​Debug checklist

​Next steps

Overview

Diagnostics API

Client-side diagnostics

Diagnostic fields

Renderer diagnostics

Common issues

Headed mode

Using CLI

Using API

Remote debugging

Enable remote debugging

Connect DevTools

Debugging tips

Playwright trace viewer

Enable trace recording

View the trace

Debugging with traces

Frame-by-frame debugging

Export frames for inspection

Seek to specific frame

Console logging

Log frame state

Log animation state

Log driver activity

Common rendering issues

Animations don’t sync

Fonts not loading

Images appear blank

Canvas shows old frame

Audio out of sync

Memory leak during preview

Performance debugging

Measure frame render time

Identify bottlenecks

Debug checklist

Next steps