Option to make images generated from a given manual seed consistent across CUDA and MPS devices
diff --git a/modules/devices.py b/modules/devices.py
index 52c3e7cd773..3bc86a6a9d1 100644
--- a/modules/devices.py
+++ b/modules/devices.py
@@ -92,14 +92,18 @@ def cond_cast_float(input):
 
 
 def randn(seed, shape):
+    from modules.shared import opts
+
     torch.manual_seed(seed)
-    if device.type == 'mps':
+    if opts.use_cpu_randn or device.type == 'mps':
         return torch.randn(shape, device=cpu).to(device)
     return torch.randn(shape, device=device)
 
 
 def randn_without_seed(shape):
-    if device.type == 'mps':
+    from modules.shared import opts
+
+    if opts.use_cpu_randn or device.type == 'mps':
         return torch.randn(shape, device=cpu).to(device)
     return torch.randn(shape, device=device)
 
diff --git a/modules/sd_samplers_common.py b/modules/sd_samplers_common.py
index a1aac7cf0aa..e6a372d52f8 100644
--- a/modules/sd_samplers_common.py
+++ b/modules/sd_samplers_common.py
@@ -60,3 +60,12 @@ def store_latent(decoded):
 
 class InterruptedException(BaseException):
     pass
+
+if opts.use_cpu_randn:
+    import torchsde._brownian.brownian_interval
+
+    def torchsde_randn(size, dtype, device, seed):
+        generator = torch.Generator(devices.cpu).manual_seed(int(seed))
+        return torch.randn(size, dtype=dtype, device=devices.cpu, generator=generator).to(device)
+
+    torchsde._brownian.brownian_interval._randn = torchsde_randn
diff --git a/modules/sd_samplers_kdiffusion.py b/modules/sd_samplers_kdiffusion.py
index e9f08518fdc..13f4567aecb 100644
--- a/modules/sd_samplers_kdiffusion.py
+++ b/modules/sd_samplers_kdiffusion.py
@@ -190,7 +190,7 @@ def randn_like(self, x):
             if noise.shape == x.shape:
                 return noise
 
-        if x.device.type == 'mps':
+        if opts.use_cpu_randn or x.device.type == 'mps':
             return torch.randn_like(x, device=devices.cpu).to(x.device)
         else:
             return torch.randn_like(x)
diff --git a/modules/shared.py b/modules/shared.py
index 5fd0eecbd1f..59b037d5897 100644
--- a/modules/shared.py
+++ b/modules/shared.py
@@ -331,6 +331,7 @@ def list_samplers():
     "comma_padding_backtrack": OptionInfo(20, "Increase coherency by padding from the last comma within n tokens when using more than 75 tokens", gr.Slider, {"minimum": 0, "maximum": 74, "step": 1 }),
     "CLIP_stop_at_last_layers": OptionInfo(1, "Clip skip", gr.Slider, {"minimum": 1, "maximum": 12, "step": 1}),
     "upcast_attn": OptionInfo(False, "Upcast cross attention layer to float32"),
+    "use_cpu_randn": OptionInfo(False, "Use CPU for random number generation to make manual seeds generate the same image across platforms. This may change existing seeds."),
 }))
 
 options_templates.update(options_section(('compatibility', "Compatibility"), {

**Describe what this pull request is trying to achieve.**

Implements an option that, when enabled, causes a given manual seed to generate identical or very similar images across both CUDA and MPS devices.

Fixes https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/9613

**Additional notes and description of your changes**

This change set makes minor modifications to existing or historical code paths contributed by @brkirch . CPU is already used for random number generation on macOS due to the MPS device's historical inability to deterministically generate random numbers.

The proposed changes add an option, `use_cpu_randn`, that will follow those code paths when the option is set, even if the GPU device is not MPS.

Enabling this option has no impact on the images an MPS device will generate, but will change a CUDA device's output to closely match MPS.

**Environment this was tested in**

 - OS: Windows 10, Linux (Arch), macOS 13.3.1
 - Browser: Chrome, Edge, Firefox
 - Graphics card: NVIDIA RTX 2080 8GB, Apple M1 Max 10cpu/32gpu

Samplers tested for similar output between CUDA and MPS: DPM++ 2M Karras, DPM++ SDE, Euler a

**Screenshots or videos of your changes**

The new option is added to the bottom of the Stable Diffusion tab in Settings.

Before addition of new option:

<img width="1152" alt="Screenshot 2023-04-19 at 01 30 48" src="https://user-images.githubusercontent.com/1689220/232975775-588031af-3ef5-4cc3-a712-0a45d348af78.png">

After addition of new option:

<img width="1153" alt="Screenshot 2023-04-19 at 01 29 40" src="https://user-images.githubusercontent.com/1689220/232975688-f98cc6b9-2929-482e-9d9a-2acdbf74e9c9.png">

(These screenshots were taken on an instance configured to use a theme from the catppuccin extension. This PR does not propose any theme-related changes.)

MPS generated image:
![00005-4044615136](https://user-images.githubusercontent.com/1689220/232977243-3bead5c0-dbd7-4235-9c0e-ac12c297ed26.png)

CUDA generated image with `use_cpu_randn` disabled:
![00047-4044615136](https://user-images.githubusercontent.com/1689220/232977042-cc1ae72e-bf1c-4456-ad61-975016d5dafb.png)

CUDA generated image with `use_cpu_randn` enabled:
![00006-4044615136](https://user-images.githubusercontent.com/1689220/232977083-d9dee76f-34af-4ab2-95e5-bf707d019549.png)
