Instructions to use YiYiXu/diff-diff-mellon with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use YiYiXu/diff-diff-mellon with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("YiYiXu/diff-diff-mellon", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
| # YiYi Notes: use this to generate mellon config | |
| # TODO: | |
| # 1. the whole process to create a custom node should be automated with something like `node.push_to_hub()` | |
| # 2. should be able to edit mellon_config.json | |
| from diffusers.modular_pipelines.node_utils import ModularNode | |
| from diffusers.modular_pipelines import ModularPipelineMixin | |
| diffdiff = ModularPipelineMixin.from_pretrained("YiYiXu/modular-diffdiff", trust_remote_code=True) | |
| # this is the complete pipeline | |
| node = ModularNode(diffdiff) | |
| node.save_mellon_config("diffdiff_mellon_config.json") | |
| # this is the denoise pipeline, to be used in the modular setting, | |
| # remove the text_encoder, decode, and ip_adapter blocks since we already have these nodes in official modular diffusers mellon | |
| diffdiff.blocks.pop("text_encoder") | |
| diffdiff.blocks.pop("decode") | |
| diffdiff.blocks.pop("ip_adapter") | |
| node = ModularNode(diffdiff, label = "DiffDiffDenoiseBlocks") | |
| node.save_mellon_config("diffdiff_denoise_mellon_config.json") | |