Fix cupy install version detection
diff --git a/.github/workflows/nv-torch19-p40.yml b/.github/workflows/nv-torch19-p40.yml
index 0c83b3a44ef8..9d21c1506b4d 100644
--- a/.github/workflows/nv-torch19-p40.yml
+++ b/.github/workflows/nv-torch19-p40.yml
@@ -45,7 +45,7 @@ jobs:
 
       - name: Install deepspeed
         run: |
-          pip install .[dev,1bit,autotuning]
+          pip install .[dev,1bit,autotuning] --no-build-isolation
           ds_report
 
       - name: Python environment
diff --git a/.github/workflows/nv-torch19-v100.yml b/.github/workflows/nv-torch19-v100.yml
index cb663b1b834d..562e961e9f3e 100644
--- a/.github/workflows/nv-torch19-v100.yml
+++ b/.github/workflows/nv-torch19-v100.yml
@@ -45,7 +45,7 @@ jobs:
 
       - name: Install deepspeed
         run: |
-          pip install .[dev,1bit,autotuning]
+          pip install .[dev,1bit,autotuning] --no-build-isolation
           ds_report
 
       - name: Python environment
diff --git a/.github/workflows/setup-venv/action.yml b/.github/workflows/setup-venv/action.yml
index dacd50b8d471..1828b2ed9eef 100644
--- a/.github/workflows/setup-venv/action.yml
+++ b/.github/workflows/setup-venv/action.yml
@@ -15,6 +15,7 @@ runs:
         python -m venv unit-test-venv
         source ./unit-test-venv/bin/activate
         python -m pip install --upgrade pip
+        pip install wheel # required after pip>=23.1
         echo PATH=$PATH >> $GITHUB_ENV # Make it so venv is inherited for other steps
       shell: bash
     - id: print-env
diff --git a/setup.py b/setup.py
index 1c7c29663700..b032bcf362c1 100755
--- a/setup.py
+++ b/setup.py
@@ -78,7 +78,12 @@ def fetch_requirements(path):
         if rocm_major <= 4:
             cupy = f"cupy-rocm-{rocm_major}-{rocm_minor}"
     else:
-        cupy = f"cupy-cuda{''.join(map(str,installed_cuda_version()))}"
+        cuda_major_ver, cuda_minor_ver = installed_cuda_version()
+        if (cuda_major_ver < 11) or ((cuda_major_ver == 11) and (cuda_minor_ver < 3)):
+            cupy = f"cupy-cuda{cuda_major_ver}{cuda_minor_ver}"
+        else:
+            cupy = f"cupy-cuda{cuda_major_ver}x"
+
     if cupy:
         extras_require['1bit'].append(cupy)
         extras_require['1bit_mpi'].append(cupy)

For later versions of cuda, we would try to install a cupy version that doesn't exist. Updating our `setup.py` to match https://docs.cupy.dev/en/stable/install.html#installing-cupy-from-pypi

Also, with the recent release of pip 23.1, our setup.py will not correctly install cupy due to isolated builds being the default now. More on that here: https://github.com/pypa/pip/issues/8559

This change has further implications for installing DeepSpeed from source. We will need to rethink what we do in `setup.py` and/or require users to install with the `--no-build-isolation` option for pip