Update supported models
diff --git a/README.md b/README.md
index bfd7f8fe87..a600271628 100644
--- a/README.md
+++ b/README.md
@@ -16,7 +16,7 @@ An open platform for training, serving, and evaluating large language model base
 
 ## Contents
 - [Install](#install)
-- [Vicuna Weights](#vicuna-weights)
+- [Model Weights](#model-weights)
 - [Inference with Command Line Interface](#inference-with-command-line-interface)
 - [Serving with Web GUI](#serving-with-web-gui)
 - [API](#api)
@@ -50,7 +50,8 @@ pip3 install --upgrade pip  # enable PEP 660 support
 pip3 install -e .
 ```
 
-## Vicuna Weights
+## Model Weights
+### Vicuna Weights
 We release [Vicuna](https://vicuna.lmsys.org/) weights as delta weights to comply with the LLaMA model license.
 You can add our delta to the original LLaMA weights to obtain the Vicuna weights. Instructions:
 
@@ -61,7 +62,7 @@ You can add our delta to the original LLaMA weights to obtain the Vicuna weights
 Weights v1.1 are only compatible with ```transformers>=4.28.0``` and ``fschat >= 0.2.0``.
 Please update your local packages accordingly. If you follow the above commands to do a fresh install, then you should get all the correct versions.
 
-### Vicuna-7B
+#### Vicuna-7B
 This conversion command needs around 30 GB of CPU RAM.
 See the "Low CPU Memory Conversion" section below if you do not have enough memory.
 ```bash
@@ -71,7 +72,7 @@ python3 -m fastchat.model.apply_delta \
     --delta-path lmsys/vicuna-7b-delta-v1.1
 ```
 
-### Vicuna-13B
+#### Vicuna-13B
 This conversion command needs around 60 GB of CPU RAM.
 See the "Low CPU Memory Conversion" section below if you do not have enough memory.
 ```bash
@@ -81,51 +82,62 @@ python3 -m fastchat.model.apply_delta \
     --delta-path lmsys/vicuna-13b-delta-v1.1
 ```
 
-### Fastchat-T5
-This model is stored in a Hugging Face [repo](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0). Simply run the line below to start chatting.
-```bash
-python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0
-```
+#### Old weights
+See [docs/vicuna_weights_version.md](docs/vicuna_weights_version.md) for all versions of weights and their differences.
 
-### Old weights
-See [docs/weights_version.md](docs/weights_version.md) for all versions of weights and their differences.
-
-
-### Low CPU Memory Conversion
+#### Low CPU Memory Conversion
 You can try these methods to reduce the CPU RAM requirement of weight conversion.
 1. Append `--low-cpu-mem` to the commands above, which will split large weight files into smaller ones and use the disk as temporary storage. This can keep the peak memory at less than 16GB.
 2. Create a large swap file and rely on the operating system to automatically utilize the disk as virtual memory.
 
+### FastChat-T5
+Simply run the line below to start chatting.
+It will automatically download the weights from a Hugging Face [repo](https://huggingface.co/lmsys/fastchat-t5-3b-v1.0). 
+
+```bash
+python3 -m fastchat.serve.cli --model-path lmsys/fastchat-t5-3b-v1.0
+```
+
 ## Inference with Command Line Interface
 
 (Experimental Feature: You can specify `--style rich` to enable rich text output and better text streaming quality for some non-ASCII content. This may not work properly on certain terminals.)
 
 <a href="https://chat.lmsys.org"><img src="assets/screenshot_cli.png" width="70%"></a>
 
+#### Supported Models
+The following models are tested:
+- Vicuna, Alpaca, LLaMA, Koala
+- [lmsys/fastchat-t5-3b-v1.0](https://huggingface.co/lmsys/fastchat-t5)
+- [databricks/dolly-v2-12b](https://huggingface.co/databricks/dolly-v2-12b)
+- [OpenAssistant/oasst-sft-1-pythia-12b](https://huggingface.co/OpenAssistant/oasst-sft-1-pythia-12b)
+- [project-baize/baize-lora-7B](https://huggingface.co/project-baize/baize-lora-7B)
+- [StabilityAI/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b)
+- [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)
+
 #### Single GPU
 The command below requires around 28GB of GPU memory for Vicuna-13B and 14GB of GPU memory for Vicuna-7B.
 See the "No Enough Memory" section below if you do not have enough memory.
 ```
-python3 -m fastchat.serve.cli --model-path /path/to/vicuna/weights
+python3 -m fastchat.serve.cli --model-path /path/to/model/weights
 ```
 
 #### Multiple GPUs
 You can use model parallelism to aggregate GPU memory from multiple GPUs on the same machine.
 ```
-python3 -m fastchat.serve.cli --model-path /path/to/vicuna/weights --num-gpus 2
+python3 -m fastchat.serve.cli --model-path /path/to/model/weights --num-gpus 2
 ```
 
 #### CPU Only
 This runs on the CPU only and does not require GPU. It requires around 60GB of CPU memory for Vicuna-13B and around 30GB of CPU memory for Vicuna-7B.
 ```
-python3 -m fastchat.serve.cli --model-path /path/to/vicuna/weights --device cpu
+python3 -m fastchat.serve.cli --model-path /path/to/model/weights --device cpu
 ```
 
 #### Metal Backend (Mac Computers with Apple Silicon or AMD GPUs)
 Use `--device mps` to enable GPU acceleration on Mac computers (requires torch >= 2.0).
 Use `--load-8bit` to turn on 8-bit compression.
 ```
-python3 -m fastchat.serve.cli --model-path /path/to/vicuna/weights --device mps --load-8bit
+python3 -m fastchat.serve.cli --model-path /path/to/model/weights --device mps --load-8bit
 ```
 Vicuna-7B can run on a 32GB M1 Macbook with 1 - 2 words / second.
 
@@ -134,10 +146,10 @@ Vicuna-7B can run on a 32GB M1 Macbook with 1 - 2 words / second.
 If you do not have enough memory, you can enable 8-bit compression by adding `--load-8bit` to commands above.
 This can reduce memory usage by around half with slightly degraded model quality.
 It is compatible with the CPU, GPU, and Metal backend.
-Vicuna-13B with 8-bit compression can run on a single NVIDIA 3090/4080/V100(16GB) GPU.
+Vicuna-13B with 8-bit compression can run on a single NVIDIA 3090/4080/T4/V100(16GB) GPU.
 
 ```
-python3 -m fastchat.serve.cli --model-path /path/to/vicuna/weights --load-8bit
+python3 -m fastchat.serve.cli --model-path /path/to/model/weights --load-8bit
 ```
 
 Besides, we are actively exploring more methods to make the model easier to run on more platforms.
@@ -158,14 +170,15 @@ This controller manages the distributed workers.
 
 #### Launch the model worker
 ```bash
-python3 -m fastchat.serve.model_worker --model-path /path/to/vicuna/weights
+python3 -m fastchat.serve.model_worker --model-path /path/to/model/weights
 ```
 Wait until the process finishes loading the model and you see "Uvicorn running on ...". You can launch multiple model workers to serve multiple models concurrently. The model worker will connect to the controller automatically.
 
 To ensure that your model worker is connected to your controller properly, send a test message using the following command:
 ```bash
-python3 -m fastchat.serve.test_message --model-name vicuna-13b
+python3 -m fastchat.serve.test_message --model-name vicuna-7b
 ```
+You will see a short output.
 
 #### Launch the Gradio web server
 ```bash
diff --git a/fastchat/serve/cli.py b/fastchat/serve/cli.py
index e93c6fb14b..bfbd676851 100644
--- a/fastchat/serve/cli.py
+++ b/fastchat/serve/cli.py
@@ -16,7 +16,7 @@
 from rich.markdown import Markdown
 from rich.live import Live
 
-from fastchat.serve.inference import chat_loop, ChatIO
+from fastchat.serve.inference import chat_loop, ChatIO, add_model_args
 
 
 class SimpleChatIO(ChatIO):
@@ -129,30 +129,7 @@ def main(args):
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser()
-    parser.add_argument(
-        "--model-path",
-        type=str,
-        default="facebook/opt-350m",
-        help="The path to the weights",
-    )
-    parser.add_argument(
-        "--device", type=str, choices=["cpu", "cuda", "mps"], default="cuda"
-    )
-    parser.add_argument(
-        "--gpus",
-        type=str,
-        default=None,
-        help="A single GPU like 1 or multiple GPUs like 0,2"
-    )
-    parser.add_argument("--num-gpus", type=str, default="1")
-    parser.add_argument(
-        "--max-gpu-memory",
-        type=str,
-        help="The maximum memory per gpu. Use a string like '13Gib'",
-    )
-    parser.add_argument(
-        "--load-8bit", action="store_true", help="Use 8-bit quantization."
-    )
+    add_model_args(parser)
     parser.add_argument(
         "--conv-template", type=str, default=None, help="Conversation prompt template."
     )
diff --git a/fastchat/serve/huggingface_api.py b/fastchat/serve/huggingface_api.py
index ba957d6865..33bbe7f5e6 100644
--- a/fastchat/serve/huggingface_api.py
+++ b/fastchat/serve/huggingface_api.py
@@ -9,7 +9,7 @@
 from transformers import AutoTokenizer, AutoModelForCausalLM
 
 from fastchat.conversation import get_default_conv_template
-from fastchat.serve.inference import load_model
+from fastchat.serve.inference import load_model, add_model_args
 
 
 @torch.inference_mode()
@@ -37,8 +37,12 @@ def main(args):
         temperature=0.7,
         max_new_tokens=1024,
     )
-    output_ids = output_ids[0][len(input_ids[0]):]
-    outputs = tokenizer.decode(output_ids, skip_special_tokens=True)
+    if model.config.is_encoder_decoder:
+        output_ids = output_ids[0]
+    else:
+        output_ids = output_ids[0][len(input_ids[0]):]
+    outputs = tokenizer.decode(output_ids, skip_special_tokens=True,
+                               spaces_between_special_tokens=False)
 
     print(f"{conv.roles[0]}: {msg}")
     print(f"{conv.roles[1]}: {outputs}")
@@ -46,24 +50,7 @@ def main(args):
 
 if __name__ == "__main__":
     parser = argparse.ArgumentParser()
-    parser.add_argument(
-        "--model-path",
-        type=str,
-        default="facebook/opt-350m",
-        help="The path to the weights",
-    )
-    parser.add_argument(
-        "--device", type=str, choices=["cpu", "cuda", "mps"], default="cuda"
-    )
-    parser.add_argument("--num-gpus", type=str, default="1")
-    parser.add_argument(
-        "--max-gpu-memory",
-        type=str,
-        help="The maximum memory per gpu. Use a string like '13Gib'",
-    )
-    parser.add_argument(
-        "--load-8bit", action="store_true", help="Use 8-bit quantization."
-    )
+    add_model_args(parser)
     parser.add_argument(
         "--conv-template", type=str, default=None, help="Conversation prompt template."
     )
diff --git a/fastchat/serve/inference.py b/fastchat/serve/inference.py
index 29d601d0b2..7804d00292 100644
--- a/fastchat/serve/inference.py
+++ b/fastchat/serve/inference.py
@@ -307,3 +307,31 @@ def chat_loop(
 
         if debug:
             print("\n", {"prompt": prompt, "outputs": outputs}, "\n")
+
+
+def add_model_args(parser):
+    parser.add_argument(
+        "--model-path",
+        type=str,
+        default="lmsys/fastchat-t5-3b-v1.0",
+        help="The path to the weights. This can be a local folder or a Hugging Face repo ID.",
+    )
+    parser.add_argument(
+        "--device", type=str, choices=["cpu", "cuda", "mps"], default="cuda",
+        help="The device type"
+    )
+    parser.add_argument(
+        "--gpus",
+        type=str,
+        default=None,
+        help="A single GPU like 1 or multiple GPUs like 0,2"
+    )
+    parser.add_argument("--num-gpus", type=str, default="1")
+    parser.add_argument(
+        "--max-gpu-memory",
+        type=str,
+        help="The maximum memory per gpu. Use a string like '13Gib'",
+    )
+    parser.add_argument(
+        "--load-8bit", action="store_true", help="Use 8-bit quantization"
+    )
diff --git a/fastchat/serve/model_worker.py b/fastchat/serve/model_worker.py
index 65aa2b726f..2688291af4 100644
--- a/fastchat/serve/model_worker.py
+++ b/fastchat/serve/model_worker.py
@@ -34,7 +34,7 @@
 import uvicorn
 
 from fastchat.constants import WORKER_HEART_BEAT_INTERVAL
-from fastchat.serve.inference import load_model, generate_stream
+from fastchat.serve.inference import load_model, generate_stream, add_model_args
 from fastchat.serve.serve_chatglm import chatglm_generate_stream
 from fastchat.utils import build_logger, server_error_msg, pretty_print_semaphore
 
@@ -219,29 +219,8 @@ async def api_get_status(request: Request):
     parser.add_argument(
         "--controller-address", type=str, default="http://localhost:21001"
     )
-    parser.add_argument(
-        "--model-path",
-        type=str,
-        default="facebook/opt-350m",
-        help="The path to the weights",
-    )
-    parser.add_argument("--model-name", type=str, help="Optional name")
-    parser.add_argument(
-        "--device", type=str, choices=["cpu", "cuda", "mps"], default="cuda"
-    )
-    parser.add_argument("--num-gpus", type=int, default=1)
-    parser.add_argument(
-        "--gpus",
-        type=str,
-        default=None,
-        help="A single GPU like 1 or multiple GPUs like 0,2"
-    )
-    parser.add_argument(
-        "--max-gpu-memory",
-        type=str,
-        help="The maximum memory per gpu. Use a string like '13Gib'",
-    )
-    parser.add_argument("--load-8bit", action="store_true")
+    add_model_args(parser)
+    parser.add_argument("--model-name", type=str, help="Optional display name")
     parser.add_argument("--limit-model-concurrency", type=int, default=5)
     parser.add_argument("--stream-interval", type=int, default=2)
     parser.add_argument("--no-register", action="store_true")
