Usage: Infinity
#6
by michaelfeil - opened
README.md
CHANGED
|
@@ -2034,7 +2034,11 @@ print(scores.tolist())
|
|
| 2034 |
# [[40.13203811645508, 25.032546997070312], [15.00684642791748, 39.937339782714844]]
|
| 2035 |
```
|
| 2036 |
|
|
|
|
| 2037 |
|
| 2038 |
-
|
| 2039 |
-
|
| 2040 |
-
|
|
|
|
|
|
|
|
|
|
|
|
| 2034 |
# [[40.13203811645508, 25.032546997070312], [15.00684642791748, 39.937339782714844]]
|
| 2035 |
```
|
| 2036 |
|
| 2037 |
+
### Usage with infinity
|
| 2038 |
|
| 2039 |
+
[Infinity, a MIT Licensed Server for embedding inference](https://github.com/michaelfeil/infinity)
|
| 2040 |
+
```
|
| 2041 |
+
docker run --gpus all -v $PWD/data:/app/.cache -e HF_TOKEN=$HF_TOKEN -p "7997":"7997" \
|
| 2042 |
+
michaelf34/infinity:0.0.68 \
|
| 2043 |
+
v2 --model-id Salesforce/SFR-Embedding-2_R --revision "91762139d94ed4371a9fa31db5551272e0b83818" --dtype bfloat16 --batch-size 4 --device cuda --engine torch --port 7997 --no-bettertransformer
|
| 2044 |
+
```
|