How to use fal/ltx2.3-audio-reactive-lora with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline from diffusers.utils import load_image, export_to_video # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("Lightricks/LTX-2.3", dtype=torch.bfloat16, device_map="cuda") pipe.load_lora_weights("fal/ltx2.3-audio-reactive-lora") prompt = "A man with short gray hair plays a red electric guitar." input_image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/guitar-man.png") image = pipe(image=input_image, prompt=prompt).frames[0] export_to_video(output, "output.mp4")
How to use fal/ltx2.3-audio-reactive-lora with LTX-2:
# Install the LTX-2 pipelines git clone https://github.com/Lightricks/LTX-2.git cd LTX-2 uv sync --frozen
# Download the weights from this repo, plus the Gemma text encoder hf download fal/ltx2.3-audio-reactive-lora --local-dir models/ltx2.3-audio-reactive-lora hf download google/gemma-3-12b-it-qat-q4_0-unquantized --local-dir models/gemma-3-12b
# Text/image-to-video with the LoRA on the HQ two-stage base pipeline uv run python -m ltx_pipelines.ti2vid_two_stages_hq \ --checkpoint-path path/to/checkpoint.safetensors \ --distilled-lora path/to/distilled_lora.safetensors 0.8 \ --spatial-upsampler-path path/to/spatial_upsampler.safetensors \ --gemma-root models/gemma-3-12b \ --lora models/ltx2.3-audio-reactive-lora/<weights>.safetensors 1.0 \ --prompt "your prompt here" \ --output-path output.mp4 # For image-to-video, add: --image path/to/image.jpg 0 0.8
