Diffusers Ruby

Ruby-first SDXL image generation powered by torch-rb. Python Diffusers is the behavioral reference, not a runtime dependency.

Supported

• Local LDM-format SDXL 1.0 base .safetensors checkpoints
• Euler Ancestral sampling
• 512×512 PNG and PPM output
• NVIDIA CUDA, Apple MPS, and CPU

Model IDs, Hub checkpoint downloads, refiners, LoRA, ControlNet, and other schedulers are not supported yet.

Requirements

• Ruby 3.2 or newer
• torch-rb 0.23.x with LibTorch 2.10.x
• An SDXL base checkpoint

torch-rb and LibTorch versions must match. See the torch-rb compatibility table.

Install on Arch Linux with NVIDIA

This setup was verified on x86-64 with an NVIDIA GeForce RTX 5070.

sudo pacman -S --needed cuda cudnn
nvidia-smi

libtorch_dir="${XDG_DATA_HOME:-$HOME/.local/share}/libtorch/2.10.0-cu128"
mkdir -p "$libtorch_dir"
curl -L --fail --retry 3 -o "$libtorch_dir/libtorch.zip" "https://download.pytorch.org/libtorch/cu128/libtorch-shared-with-deps-2.10.0%2Bcu128.zip"
unzip -q "$libtorch_dir/libtorch.zip" -d "$libtorch_dir"

bundle config set --local build.torch-rb "--with-torch-dir=$libtorch_dir/libtorch --with-cuda-dir=/opt/cuda --with-cudnn-dir=/usr"
bundle install

Verify CUDA from Ruby:

bundle exec ruby -e 'require "torch"; puts Torch::CUDA.available?; puts Torch.ones([2, 2], device: "cuda").sum.item'

Expected output is true and 4.0.

For macOS or CPU-only installation, download the matching LibTorch 2.10.x build and follow the torch-rb installation instructions.

Add the model

Download the 6.94 GB official checkpoint and place it here:

tmp/models/sd_xl_base_1.0.safetensors

The checkpoint is ignored by Git. The first generation downloads two tokenizer JSON files to ~/.cache/diffusers-ruby.

Generate an image

bin/diffusers --model=tmp/models/sd_xl_base_1.0.safetensors --steps=30 --positive-prompt="a tiny red panda writing Ruby" --negative-prompt="blurry, low quality" --cfg=5

Output defaults to tmp/sdxl.png. Run bin/diffusers --help for --output and --seed options.

Ruby API

require "diffusers"

pipeline = Diffusers::Pipelines::SDXL::TextToImage.from_single_file(
  "tmp/models/sd_xl_base_1.0.safetensors",
  scheduler: :euler_a,
  device: :auto,
  dtype: :auto
)

result = pipeline.call(
  prompt: "a tiny red panda writing Ruby",
  negative_prompt: "blurry, low quality",
  num_inference_steps: 30,
  guidance_scale: 5.0,
  seed: 0
)

result.save("tmp/sdxl.png")

:auto selects MPS with float16, then CUDA with float16, then CPU with float32. The VAE remains float32. Use output_type: :latent to skip image decoding.

Benchmark

See BENCHMARK.md for the benchmark method, RTX 5070 results, and reproduction command.

Troubleshooting

LibTorch not found

Run bundle config get build.torch-rb. Confirm that the configured directory contains libtorch/include and libtorch/lib.

CUDA out of memory

Run nvidia-smi and stop other compute workloads. ComfyUI can use enough VRAM to prevent SDXL from running on a 12 GB card.

Development

mise install
bundle install
bundle exec rake test

Run the checkpoint-gated integration test with:

SDXL_CHECKPOINT=tmp/models/sd_xl_base_1.0.safetensors bundle exec ruby -Itest test/test_sdxl_integration.rb