Skip to content

StableDiffusionXLPipeline.from_single_file Error #7389

Closed
Closed
StableDiffusionXLPipeline.from_single_file Error#7389
Labels
bugSomething isn't workingstaleIssues that haven't received updates

Description

@cendelian
cendelian
opened on Mar 19, 2024

Describe the bug

version: 0.27.0, 0.27.1
when use StableDiffusionXLPipeline.from_single_file,error occur:

  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/diffusers/pipelines/stable_diffusion_xl/pipeline_stable_diffusion_xl.py", line 383, in encode_prompt
    prompt_embeds = text_encoder(text_input_ids.to(device), output_hidden_states=True)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 822, in forward
    return self.text_model(
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 730, in forward
    hidden_states = self.embeddings(input_ids=input_ids, position_ids=position_ids)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/transformers/models/clip/modeling_clip.py", line 229, in forward
    position_embeddings = self.position_embedding(position_ids)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/modules/sparse.py", line 162, in forward
    return F.embedding(
  File "aaa/anaconda/envs/sd-webui/lib/python3.10/site-packages/torch/nn/functional.py", line 2210, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: Expected tensor for argument #1 'indices' to have one of the following scalar types: Long, Int; but got torch.cuda.HalfTensor instead (while checking arguments for embedding)

but version 0.26.3 is OK

Reproduction

pipe = StableDiffusionXLPipeline.from_single_file('xxxxxxx.safetensors',
 original_config_file='aaaa/sd_xl_base.yaml', 
 torch_dtype=torch.float16, 
 local_files_only=True,
 use_safetensors=True,
 add_watermarker=False,
#  safety_checker=None,
#  requires_safety_checker=False
 )
print("22222222222222")

pipe.to('cuda')
prompt = "black denim,long jeans,1girl,datou,"
negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)'


# sdxl
image1 = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=50,
    eta=1.0,
).images[0]

image1.save("test.png")

Logs

No response

System Info

diffuser 0.27.0 0.27.1

Who can help?

No response

Activity

  1. cendelian
    added
    bugSomething isn't working
    on Mar 19, 2024

  2. yiyixuxu commented on Mar 19, 2024

    @yiyixuxu
    yiyixuxu
    on Mar 19, 2024
    Collaborator

    sorry I can't reproduce this, this runs for me

    the config file is https://raw.githubusercontent.com/Stability-AI/generative-models/main/configs/inference/sd_xl_base.yaml

    from diffusers import StableDiffusionXLPipeline
import torch

repo_id = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors"
pipe = StableDiffusionXLPipeline.from_single_file(repo_id,
 original_config_file='sd_xl_base.yaml', 
 torch_dtype=torch.float16, 
 local_files_only=True,
 use_safetensors=True,
 add_watermarker=False,
 )

pipe.enable_model_cpu_offload()
prompt = "black denim,long jeans,1girl,datou,"
negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)'


# sdxl
image1 = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=50,
    eta=1.0,
).images[0]

image1.save("yiyi_test_out.png")

  3. cendelian commented on Mar 20, 2024

    @cendelian
    cendelian
    on Mar 20, 2024
    Author

    sorry I can't reproduce this, this runs for me

    the config file is https://raw.githubusercontent.com/Stability-AI/generative-models/main/configs/inference/sd_xl_base.yaml

    from diffusers import StableDiffusionXLPipeline
import torch

repo_id = "https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/sd_xl_base_1.0.safetensors"
pipe = StableDiffusionXLPipeline.from_single_file(repo_id,
 original_config_file='sd_xl_base.yaml', 
 torch_dtype=torch.float16, 
 local_files_only=True,
 use_safetensors=True,
 add_watermarker=False,
 )

pipe.enable_model_cpu_offload()
prompt = "black denim,long jeans,1girl,datou,"
negative_prompt = '(worst quality:2.1),(low quality:2.1),(normal quality:2.1)'


# sdxl
image1 = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    num_inference_steps=50,
    eta=1.0,
).images[0]

image1.save("yiyi_test_out.png")

    what's your version? 0.26.3 is OK, but 0.27.0/0.27.1/0.27.2 occur error
    sd_xl_base_1.0.safetensors
    sd_xl_base.yaml

  4. sayakpaul commented on Mar 20, 2024

    @sayakpaul
    sayakpaul
    on Mar 20, 2024
    Member

    Unable to reproduce: https://colab.research.google.com/gist/sayakpaul/62e5e57f34161122b8a1439906b84c51/scratchpad.ipynb.

  5. cbstyle commented on Apr 9, 2024

    @cbstyle
    cbstyle
    on Apr 9, 2024

    Have you solved the problem?

  6. github-actions commented on May 3, 2024

    @github-actions
    github-actionsBot
    on May 3, 2024 – with GitHub Actions
    Contributor

    This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

    Please note that issues that do not follow the contributing guidelines are likely to be ignored.

  7. github-actions
    added
    staleIssues that haven't received updates
    on May 3, 2024

  8. yiyixuxu commented on May 3, 2024

    @yiyixuxu
    yiyixuxu
    on May 3, 2024
    Collaborator

    closing now since we are not able to reproduce on our end
    if this is still an issue, please let us know and we will reopen it!

  9. yiyixuxu
    closed this as completedon May 3, 2024

  10. eija06 commented on May 9, 2024

    @eija06
    eija06
    on May 9, 2024

    I am able to reproduce it with the following minimal code:

    from diffusers import StableDiffusionXLPipeline
    import torch

    pipeline = StableDiffusionXLPipeline.from_single_file(
    "/workspace/stable-diffusion-webui/models/Stable-diffusion/realismEngineSDXL_v30VAE.safetensors",
    torch_dtype=torch.float16
    ).to("cuda")

    image = pipeline(prompt="photo").images[0]

    The venv is
    absl-py==1.4.0
    accelerate==0.21.0
    addict==2.4.0
    aenum==3.1.15
    aiofiles==23.2.1
    aiohttp==3.8.5
    aiosignal==1.3.1
    altair==5.1.1
    antlr4-python3-runtime==4.9.3
    anyio==3.7.1
    argon2-cffi==23.1.0
    argon2-cffi-bindings==21.2.0
    arrow==1.2.3
    asttokens==2.2.1
    async-lru==2.0.4
    async-timeout==4.0.3
    attrs==23.1.0
    av==10.0.0
    Babel==2.12.1
    backcall==0.2.0
    basicsr==1.4.2
    beautifulsoup4==4.12.2
    bleach==6.0.0
    blendmodes==2022
    boltons==23.0.0
    cachetools==5.3.1
    certifi==2023.7.22
    cffi==1.15.1
    chardet==5.2.0
    charset-normalizer==3.2.0
    clean-fid==0.1.35
    click==8.1.7
    clip==1.0
    cmake==3.25.0
    comm==0.1.4
    contourpy==1.1.0
    cssselect2==0.7.0
    cycler==0.11.0
    debugpy==1.6.7.post1
    decorator==5.1.1
    defusedxml==0.7.1
    deprecation==2.1.0
    diffusers==0.27.2
    dynamicprompts==0.31.0
    einops==0.4.1
    exceptiongroup==1.1.3
    executing==1.2.0
    facexlib==0.3.0
    fake-useragent==1.5.1
    fastapi==0.94.0
    fastjsonschema==2.18.0
    ffmpy==0.3.1
    filelock==3.9.0
    filterpy==1.4.5
    flatbuffers==23.5.26
    fonttools==4.42.1
    fqdn==1.5.1
    frozenlist==1.4.0
    fsspec==2023.6.0
    ftfy==6.1.1
    future==0.18.3
    fvcore==0.1.5.post20221221
    gdown==4.7.1
    gfpgan==1.3.8
    gitdb==4.0.10
    GitPython==3.1.32
    google-auth==2.22.0
    google-auth-oauthlib==1.0.0
    gradio==3.41.2
    gradio_client==0.5.0
    grpcio==1.57.0
    h11==0.12.0
    httpcore==0.15.0
    httpx==0.24.1
    huggingface-hub==0.23.0
    idna==3.4
    imageio==2.31.2
    imageio-ffmpeg==0.4.8
    importlib-metadata==6.8.0
    importlib-resources==6.0.1
    inflection==0.5.1
    iopath==0.1.10
    ipykernel==6.25.1
    ipython==8.14.0
    ipywidgets==8.1.0
    isoduration==20.11.0
    jedi==0.19.0
    Jinja2==3.1.2
    json5==0.9.14
    jsonmerge==1.8.0
    jsonpointer==2.4
    jsonschema==4.19.0
    jsonschema-specifications==2023.7.1
    jupyter-events==0.7.0
    jupyter-lsp==2.2.0
    jupyter_client==8.3.1
    jupyter_core==5.3.1
    jupyter_server==2.7.3
    jupyter_server_terminals==0.4.4
    jupyterlab==4.0.5
    jupyterlab-pygments==0.2.2
    jupyterlab-widgets==3.0.8
    jupyterlab_server==2.24.0
    kiwisolver==1.4.5
    kornia==0.6.7
    lark==1.1.2
    lazy_loader==0.3
    lightning-utilities==0.9.0
    lit==15.0.7
    llvmlite==0.40.1
    lmdb==1.4.1
    lpips==0.1.4
    lxml==4.9.3
    lycoris-lora==2.2.0.post3
    Markdown==3.4.4
    markdown-it-py==3.0.0
    MarkupSafe==2.1.3
    matplotlib==3.7.2
    matplotlib-inline==0.1.6
    mdurl==0.1.2
    mediapipe==0.10.3
    mistune==3.0.1
    mpmath==1.2.1
    multidict==6.0.4
    nbclient==0.8.0
    nbconvert==7.8.0
    nbformat==5.9.2
    nest-asyncio==1.5.7
    networkx==3.0
    notebook_shim==0.2.3
    numba==0.57.1
    numexpr==2.8.5
    numpy==1.23.5
    oauthlib==3.2.2
    omegaconf==2.2.3
    open-clip-torch==2.20.0
    opencv-contrib-python==4.8.0.76
    opencv-python==4.8.0.76
    orjson==3.9.5
    overrides==7.4.0
    packaging==23.1
    pandas==2.1.0
    pandocfilters==1.5.0
    parso==0.8.3
    pexpect==4.8.0
    pickleshare==0.7.5
    piexif==1.1.3
    Pillow==9.5.0
    PIMS==0.6.1
    platformdirs==3.10.0
    portalocker==2.7.0
    prometheus-client==0.17.1
    prompt-toolkit==3.0.39
    protobuf==3.20.0
    psutil==5.9.5
    ptyprocess==0.7.0
    pure-eval==0.2.2
    py-cpuinfo==9.0.0
    pyasn1==0.5.0
    pyasn1-modules==0.3.0
    pycparser==2.21
    pydantic==1.10.12
    pydub==0.25.1
    Pygments==2.16.1
    pyparsing==3.0.9
    PySocks==1.7.1
    python-dateutil==2.8.2
    python-json-logger==2.0.7
    python-multipart==0.0.6
    pytorch-lightning==1.9.4
    pytz==2023.3
    PyWavelets==1.4.1
    PyYAML==6.0.1
    pyzmq==25.1.1
    realesrgan==0.3.0
    referencing==0.30.2
    regex==2023.8.8
    reportlab==4.0.4
    requests==2.31.0
    requests-oauthlib==1.3.1
    resize-right==0.0.2
    rfc3339-validator==0.1.4
    rfc3986-validator==0.1.1
    rich==13.5.2
    rpds-py==0.10.0
    rsa==4.9
    safetensors==0.3.1
    scikit-image==0.21.0
    scipy==1.11.2
    seaborn==0.13.2
    semantic-version==2.10.0
    Send2Trash==1.8.2
    sentencepiece==0.1.99
    six==1.16.0
    slicerator==1.1.0
    smmap==5.0.0
    sniffio==1.3.0
    sounddevice==0.4.6
    soupsieve==2.4.1
    stack-data==0.6.2
    starlette==0.26.1
    svglib==1.5.1
    sympy==1.11.1
    tabulate==0.9.0
    tb-nightly==2.15.0a20230831
    tensorboard-data-server==0.7.1
    termcolor==2.3.0
    terminado==0.17.1
    thop==0.1.1.post2209072238
    tifffile==2023.8.30
    timm==0.9.2
    tinycss2==1.2.1
    tokenizers==0.13.3
    tomesd==0.1.3
    toml==0.10.2
    tomli==2.0.1
    toolz==0.12.0
    torch==2.0.1+cu118
    torchaudio==2.0.2+cu118
    torchdiffeq==0.2.3
    torchmetrics==1.1.1
    torchsde==0.2.5
    torchvision==0.15.2+cu118
    tornado==6.3.3
    tqdm==4.66.1
    traitlets==5.9.0
    trampoline==0.1.2
    transformers==4.30.2
    triton==2.0.0
    typing_extensions==4.7.1
    tzdata==2023.3
    ultralytics==8.2.8
    uri-template==1.3.0
    urllib3==1.26.16
    uvicorn==0.23.2
    wcwidth==0.2.6
    webcolors==1.13
    webencodings==0.5.1
    websocket-client==1.6.2
    websockets==11.0.3
    Werkzeug==2.3.7
    widgetsnbextension==4.0.8
    xformers==0.0.21
    yacs==0.1.8
    yapf==0.40.1
    yarl==1.9.2
    zipp==3.16.2
    ZipUnicode==1.1.1

    I am NOT getting this error with a fresh venv installed with the most recent versions available to pip.
    So you could close this bug report as fixed - but you might get this report again, because the venv posted above is in active use on many machines - it is the venv that stable diffusion A1111 webui creates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingstaleIssues that haven't received updates

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions