Convert any SD model into inpaint model

jysung · January 18, 2023, 8:38pm

AUTOMATIC1111/stable-diffusion-webui/blob/master/modules/sd_hijack_inpainting.py

import os
import torch

from einops import repeat
from omegaconf import ListConfig

import ldm.models.diffusion.ddpm
import ldm.models.diffusion.ddim
import ldm.models.diffusion.plms

from ldm.models.diffusion.ddpm import LatentDiffusion
from ldm.models.diffusion.plms import PLMSSampler
from ldm.models.diffusion.ddim import DDIMSampler, noise_like


@torch.no_grad()
def p_sample_plms(self, x, c, t, index, repeat_noise=False, use_original_steps=False, quantize_denoised=False,
                  temperature=1., noise_dropout=0., score_corrector=None, corrector_kwargs=None,
                  unconditional_guidance_scale=1., unconditional_conditioning=None, old_eps=None, t_next=None, dynamic_threshold=None):
    b, *_, device = *x.shape, x.device

This file has been truncated. show original

Automatic 1111 has an ability to hijack any model for inpainting.
Has anyone achieved it using diffusers?

mbug90 · March 18, 2023, 8:36pm

What is this feature and how do I use it?

williamberman · March 20, 2023, 11:55pm

We explicitly don’t support generic hijacking of pipelines for inpainting, but more than happy to merge community pipelines in line with it

Topic		Replies	Views
Converting diffusers model to inpaint model? 🧨 Diffusers	1	1157	January 18, 2023
SD XL Multi ControlNet Inpainting in diffusers 🧨 Diffusers	0	2129	September 11, 2023
Finetuning Stable Diffusion inpainting checkpoint Models	1	782	July 20, 2023
Compel doesn't work with the new Inpainting SDXL model 🧨 Diffusers	1	491	February 12, 2024
SDXL inpainting checkpoint generating dark lighting samples 🧨 Diffusers	0	586	January 19, 2024

Convert any SD model into inpaint model

Related topics