[BUG FIX] Deadlock issues when using multiprocessing
diff --git a/llama_index/ingestion/pipeline.py b/llama_index/ingestion/pipeline.py
index 6e84cc4b3d053..fee44d6b20694 100644
--- a/llama_index/ingestion/pipeline.py
+++ b/llama_index/ingestion/pipeline.py
@@ -447,7 +447,7 @@ def run(
                     "Setting `num_workers` down to the maximum CPU count."
                 )
 
-            with multiprocessing.Pool(num_workers) as p:
+            with multiprocessing.get_context("spawn").Pool(num_workers) as p:
                 node_batches = self._node_batcher(
                     num_batches=num_workers, nodes=nodes_to_run
                 )
diff --git a/llama_index/readers/file/base.py b/llama_index/readers/file/base.py
index 1c8664b89ecde..7f72d9406e54a 100644
--- a/llama_index/readers/file/base.py
+++ b/llama_index/readers/file/base.py
@@ -361,7 +361,7 @@ def load_data(
                     "Specified num_workers exceed number of CPUs in the system. "
                     "Setting `num_workers` down to the maximum CPU count."
                 )
-            with multiprocessing.Pool(num_workers) as p:
+            with multiprocessing.get_context("spawn").Pool(num_workers) as p:
                 results = p.starmap(
                     SimpleDirectoryReader.load_file,
                     zip(

# Description

- Multiprocessing context by default "fork"'s (except on Windows)
- This can create deadlock issue if fork is done at wrong time and worker processor is trying to acquire lock
- To fix, we set context to "spawn" which will start the processor by using a fresh python interpretor rather than fork.

For more information see [here](https://pythonspeed.com/articles/python-multiprocessing/) and [here](https://stackoverflow.com/questions/43818519/what-is-the-meaning-of-context-argument-in-multiprocessing-pool-pool)

Fixes #10104 

## Type of Change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

# How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

- [x] Ran previous parallel processing nbs and confirmed similar wall times are achieved under "spawn" context
- [x] I stared at the code and made sure it makes sense