I’m getting the following error when I try to run interference:
…
#instentiate llm
self.llm = HuggingFaceEndpoint(
repo_id="...",
task="text-generation",
max_new_tokens=100,
temperature=0.6
)
I changed my API key but the problem still persists.
With fine-grained tokens (the default), that error often occurs because permissions were not granted. In that case, using a read token makes it easier to isolate the issue.
Other possible causes include outdated LangChain-related libraries or subtle changes in how the HF API is called.
This 403 usually means the request reached Hugging Face’s Inference Providers layer, but the credential actually used for that request is not allowed to call Inference Providers. In your case, that is the most likely reading because HuggingFaceEndpoint(repo_id=...) is the LangChain path for Hugging Face serverless Inference Providers or dedicated Inference Endpoints, and endpoint_url is the separate path used when you want to point directly at a specific endpoint. (LangChain Document)
The important part is this phrase:
“does not have sufficient permissions to call Inference Providers”
That is not the same as:
It means Hugging Face recognized an authentication method, but authorization failed at the Inference Providers layer. Hugging Face’s current docs explicitly say provider-backed inference requires a fine-grained token with “Make calls to Inference Providers” permission. (Hugging Face)
Because there are two separate questions:
Many people only solve question 1.
Hugging Face documents that HF_TOKEN can authenticate your session and that environment-variable authentication has priority over the token stored on your machine. LangChain’s Hugging Face docs, meanwhile, commonly use HUGGINGFACEHUB_API_TOKEN. So it is very easy to rotate one token and still have the process use another one from a notebook secret, shell environment, cached CLI login, or old runtime state. (Hugging Face)
This is the top cause.
Hugging Face’s Inference Providers docs are explicit: create a fine-grained token with “Make calls to Inference Providers” permission. A token that is good enough for downloading models or reading Hub content can still fail for provider-backed inference if that permission is missing. Hugging Face’s token docs also say User Access Tokens are the preferred auth method, and organization API tokens are deprecated. (Hugging Face)
This is nearly as common as cause 1.
Typical hidden sources are:
HF_TOKENHUGGINGFACEHUB_API_TOKENhf auth loginHugging Face states that HF_TOKEN via environment variable or secret takes priority over the stored machine token. It also documents hf auth switch and hf auth list for managing multiple saved tokens. (Hugging Face)
repo_id=..., so LangChain is taking the provider-backed pathThis matters because it changes the authentication requirement.
If you pass repo_id, LangChain is working through Hugging Face’s model-routing/inference layer. If you intended to hit a self-hosted TGI server, a reverse proxy, or a dedicated endpoint you control directly, the correct knob is usually endpoint_url=..., not repo_id=.... LangChain’s docs separate these two modes, and a LangChain bug report shows that local TGI users ran into auth confusion precisely because the integration tried to authenticate with Hugging Face Hub even when the user expected purely local serving. (LangChain Document)
A lot of older tutorials and snippets were written before the current Hugging Face provider model and before the langchain_huggingface partner package became the recommended integration. Hugging Face and LangChain announced that partner package specifically to track the newer Hugging Face APIs more closely. LangChain’s current docs also point users to langchain-huggingface, not older community-only patterns. (Hugging Face)
This is not your current error, but it is a very common next error.
There is a recent LangChain issue where HuggingFaceEndpoint failed because the selected model/provider combination did not support text-generation; the provider only exposed it as conversational. So solving the 403 may reveal a second issue related to task support, not credentials. (GitHub)
Given this code:
self.llm = HuggingFaceEndpoint(
repo_id="...",
task="text-generation",
max_new_tokens=100,
temperature=0.6
)
the most likely explanation is:
The wording of the error points more strongly to authorization scope than to model/task mismatch as the first failure.
Create a new fine-grained User Access Token and enable:
That is the permission Hugging Face documents for provider-backed inference. Fine-grained tokens are also the recommended production choice. (Hugging Face)
During debugging, remove ambiguity.
Set both environment variables to the same token in the same process:
import os
HF = "hf_your_new_fine_grained_token_here"
os.environ["HF_TOKEN"] = HF
os.environ["HUGGINGFACEHUB_API_TOKEN"] = HF
This aligns the main Hugging Face auth path (HF_TOKEN) with the LangChain convention (HUGGINGFACEHUB_API_TOKEN). Hugging Face docs cover HF_TOKEN; LangChain docs cover HUGGINGFACEHUB_API_TOKEN. (Hugging Face)
Do not guess. Check.
Hugging Face documents whoami(token=...) and the CLI auth commands. Use them to verify the runtime identity before you touch the model code again. (Hugging Face)
import os
from huggingface_hub import whoami
token = os.getenv("HF_TOKEN") or os.getenv("HUGGINGFACEHUB_API_TOKEN")
print("Token present:", bool(token))
print(whoami(token=token))
And from a shell:
hf auth list
hf auth switch
hf auth whoami
If whoami shows the wrong account, you already found the root cause. If it shows the expected account and the 403 still remains, the token almost certainly lacks the required provider permission. (Hugging Face)
This is mundane but important.
If your code is running in Jupyter, Colab, VS Code, Streamlit, Docker, or a long-lived backend process, an old environment variable or cached state may still be active even after you changed secrets. Hugging Face’s docs note that environment-variable auth and secrets override stored auth, which is exactly why stale runtime state causes confusion. (Hugging Face)
Use the current package family, not a tutorial-era mix.
pip install -U langchain langchain-huggingface huggingface_hub
This is not a magic fix by itself, but it reduces the chance that you are following older endpoint or auth behavior. The current recommended integration is langchain-huggingface, maintained as the Hugging Face × LangChain partner package. (Hugging Face)
Try an explicit provider choice or provider="auto" if you are on the current LangChain path:
from langchain_huggingface import HuggingFaceEndpoint
llm = HuggingFaceEndpoint(
repo_id="your-model-id",
task="text-generation",
provider="auto",
max_new_tokens=100,
temperature=0.6,
)
LangChain’s current examples show provider="auto" and named providers. If a different error appears next saying the model is not supported for text-generation, that means the token issue is fixed and you are now dealing with provider/task compatibility. (LangChain Document)
Run this sequence in order.
Create a new fine-grained token with Make calls to Inference Providers. (Hugging Face)
Set both:
HF_TOKENHUGGINGFACEHUB_API_TOKEN (Hugging Face)Verify identity with:
whoami(token=...)hf auth whoamihf auth list (Hugging Face)Restart the notebook, shell, app server, or container. (Hugging Face)
Retry HuggingFaceEndpoint(...).
If the 403 disappears but a new error says something like “model not supported for task text-generation,” switch to a provider/task-compatible model or integration style. (GitHub)
There are really two separate Hugging Face worlds that people mix up:
This is what your current repo_id=... usage most likely invokes. It needs Inference Providers permissions. (LangChain Document)
This is where you point at your own endpoint_url, such as a local TGI server or dedicated endpoint URL. In that world, the auth story can be different, and LangChain has had issues where local users were still pushed through Hugging Face-style authentication logic unexpectedly. (LangCain Reference)
That distinction is why the exact constructor arguments matter.
For your specific error, the strongest diagnosis is:
text-generation. (GitHub)temperature, max_new_tokens, or the prompt itself. Those do not match the error shape. (Hugging Face)