How to make 2 ai vtubers talk to eachother in vtube studio?

Research
1

So um… i kinda need help on This I keep having a issue with The mouth parameter on two vtuber models I’m trying to make it so the model only uses the code of the terminal it is currently using

Such as “WAFFLE LOVING GOOBER → 8001 → Api → “MouthOpen”

This works normally it’s just a pain to figure out how to work with multiple models

Such as “hyori“ → 8002 → Api → MouthOpen”

Such as “WAFFLE LOVING GOOBER → 8001 → Api → MouthOpen”

But then i have a issue with the token files and it’s a whole pain I’ve GOTTEN it too work But only for short period or on accident And the token Files Always end up breaking afterwards.

My question is should i just make a other input Param for the mouth for the extra ai or do something else?

1 Like
2

Hmm…

For your setup with two VTube Studio instances on different ports, the best fix is not “make another mouth input parameter” first. The better fix is:

  • one VTube Studio instance per AI
  • one websocket client per instance
  • one token file per client
  • one stable plugin identity per client
  • one mouth stream sent only to that instance’s socket

That matches how VTube Studio is designed. The API starts on port 8001 by default, and if that port is already taken, the next instance moves to 8002, then 8003, and so on. VTube Studio also exposes a unique instanceID and windowTitle for each running instance, so multi-instance control is a normal supported use case. (GitHub)

The core problem

You are dealing with three separate layers at once:

  1. instance routing
    Which VTube Studio window are you actually connected to.

  2. authentication state
    Which saved token belongs to which plugin identity.

  3. parameter ownership
    Which process is currently allowed to control MouthOpen.

When those three are not kept separate, the result looks random. Mouth works for a bit, then stops. One model moves when the other should. A token seems fine, then “breaks.” That pattern matches the official API behavior very closely. (GitHub)

Why the token files feel broken

VTube Studio tokens are not generic. The official API says you only need to request a token once, then reuse it on later sessions. But pluginName and pluginDeveloper in the authentication request must match the values used when the token was created, or authentication fails. (GitHub)

That means these patterns will break things:

  • both AI clients writing to the same auth-token.txt
  • changing the plugin name during testing
  • changing the developer name during testing
  • re-requesting tokens over and over instead of reusing the saved one

This is also why wrapper libraries emphasize persistent token storage. VTubeStudioJS requires authTokenGetter and authTokenSetter to persist the token, and pyvts explicitly reads and writes a token file for future runs. (GitHub)

Why the mouth works “only for a short period”

That part is also documented.

When you use InjectParameterDataRequest, VTube Studio says you must re-send data for a parameter at least once every second or that parameter is considered “lost,” and control falls back to whatever was controlling it before, or to default if nothing else is controlling it. (GitHub)

VTube Studio also says that only one API plugin can write to one parameter at a time in normal "set" mode. If another plugin is already controlling that parameter, an error is returned. Only "add" mode can be shared by multiple plugins, and that is not what you usually want for lipsync. (GitHub)

So if either of these happens:

  • your client stops sending MouthOpen often enough
  • the wrong client touches MouthOpen in the same instance
  • some reconnect logic briefly grabs the same parameter

then the behavior will feel unstable even though VTube Studio is behaving normally. (GitHub)

Should you add another mouth input parameter

For two separate VTube Studio instances, usually no.

If you have:

  • WAFFLE LOVING GOOBER on port 8001
  • hyori on port 8002

then each instance already has its own copy of MouthOpen. The issue is almost certainly not that both models need separate mouth parameters. The issue is that the routing and auth are not isolated enough. VTube Studio also gives you CurrentModelRequest, which returns modelLoaded, modelName, and modelID, so your code can verify the loaded model before it starts sending mouth values. (GitHub)

Adding a custom mouth parameter becomes useful mainly in this different case:

  • both characters are inside the same VTube Studio instance
  • or you intentionally want separate rig-level blending logic

That is because injected parameters are used by the loaded model and any loaded Live2D items in that same instance. (GitHub)

There is also a downside to custom parameters. VTube Studio stores them in custom_parameters.json, and if a plugin’s auth token is revoked, the custom parameters created by that plugin are deleted. So if your token handling is already unstable, custom mouth parameters can make the system more fragile, not less. (GitHub)

What I would do in your case

I would keep the standard MouthOpen and fix the architecture.

Good architecture

AI 1

  • connects only to ws://127.0.0.1:8001
  • uses plugin identity like Goober Controller / YourName
  • stores token in tokens/goober.token

AI 2

  • connects only to ws://127.0.0.1:8002
  • uses plugin identity like Hyori Controller / YourName
  • stores token in tokens/hyori.token

Then each client should do this on startup:

  1. connect to its assigned port
  2. authenticate using its own saved token
  3. if auth fails, request a new token and save it to that client’s own file
  4. call CurrentModelRequest
  5. verify the loaded modelName or modelID is the expected one
  6. only then start sending MouthOpen updates
  7. keep sending while that AI is speaking (GitHub)

That is the clean solution.

What not to do

Do not do this:

  • one shared token file for both AIs
  • one generic plugin name reused across experiments
  • “request a new token every run”
  • “send mouth once and assume it sticks”
  • “add a second mouth parameter before isolating ports and auth”

Those choices are exactly the kind of thing that creates the unstable behavior you described. (GitHub)

The easier and usually more stable option

For AI VTubers, the most practical answer is often:

do not drive MouthOpen manually at all.
Route each AI’s TTS audio into its own VTube Studio instance and let VTube Studio do the lipsync.

VTube Studio officially supports microphone-based lipsync. It can derive mouth movement from audio with VoiceVolume, VoiceVolumePlusMouthOpen, and the vowel parameters VoiceA, VoiceI, VoiceU, VoiceE, and VoiceO. The official docs recommend Advanced Lipsync rather than the legacy simple mode. (GitHub)

This is also how a lot of AI VTuber projects are set up in practice. The Neuro project routes TTS output into VTube Studio through a virtual audio cable and lets VTube Studio handle lipsync. The vtuber-waifu project tells users to capture the program’s audio with Virtual Cable and use that as VTube Studio microphone input. MITSUHA gives a similar VB-Cable setup. (GitHub)

For your use case, that approach is often better because:

  • speech audio is already the truth for timing
  • you do not need to stream mouth values constantly
  • you avoid parameter ownership fights
  • you reduce the amount of mouth-specific code

That is the path I would choose first if your goal is simply “make two AI characters talk reliably.” (GitHub)

One more thing to check

Sometimes the code is fine and the model mapping is the real issue.

VTube Studio’s model settings say each output Live2D parameter can only be chosen once. They also note that if a parameter appears to move in config but the model does not visibly move, the likely cause is an expression, animation, or physics system overwriting it. (GitHub)

So for each model, verify that:

  • MouthOpen is mapped to the intended Live2D mouth-open parameter
  • that output parameter is not mapped twice
  • no expression or motion is locking the mouth
  • no physics setup is masking the visible movement (GitHub)

My direct recommendation

For your exact case, do this:

Best overall

Use two VTube Studio instances and let each one handle lipsync from its own routed TTS audio. Keep API usage for expressions, hotkeys, scene control, or model control. (GitHub)

If you want to keep API mouth control

Still use two instances, but give each AI:

  • its own port
  • its own plugin identity
  • its own token file
  • a model check with CurrentModelRequest
  • a continuous mouth-update loop while speaking (GitHub)

Do not start by adding another mouth parameter

That is only the right move if both characters are in the same VTube Studio instance or you specifically want custom rig logic. In your two-port setup, it is probably solving the wrong problem. (GitHub)

3

I might do Virtual Cable But i’m not using existing plugins like VTS Desktop Audio Plugin by Lua Lucky as it has major issues where the mouth will open TO ANY sound unlike the Api I’m using That has it isolated to the sound of the terminal tts

1 Like
4

Also Would it be possible to get it to work with the same port for vtube studios api and A custom input For the second terminal So it does not interfere with the first terminal?

(terminal 1) 8001 —> mouth open

(terminal 2)8001 —> mouth open 1 (custom param)

you Said the main issue is the token files would this cause more or less issues currently?

i am using two token files currently but right now i am unsure if the issue is because a issue on my part

1 Like
5

Hmm… When multiple programs need to communicate with each other, I think it’s better to use either a separate server process or a separate port in general.

Of course, there are cases where this isn’t feasible due to specific constraints—such as when opening an external port is difficult or when you want to conserve VRAM—but…

Separating them results in a cleaner implementation and makes it easier to coexist with other services.

Yes. Technically possible, but only if both terminals are talking to the same VTube Studio instance on 8001. In VTube Studio, port 8001 is one websocket server. If another VTS instance is already using that port, the next instance moves to 8002, then higher. So terminal 1 -> 8001 and terminal 2 -> 8001 means both terminals are controlling the same VTS window, not two separate VTS windows. (GitHub)

That means your design

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8001 → CustomMouthParam

can work only as a one-instance design. VTS explicitly allows plugins to create custom parameters and to feed data into either default or custom parameters. Those parameters then become inputs for the loaded model and any loaded Live2D items in that same instance. (GitHub)

So the real answer is:

  • two VTS windows: use 8001 and 8002
  • one VTS window: same port is possible, but the second terminal must control a different parameter, not the same one. (GitHub)

Will the two terminals interfere with each other on the same port?

They can, but not automatically. VTS allows multiple plugins to be connected at once. The API even reports connectedPlugins in StatisticsResponse, so multiple clients on one instance are a supported case. The interference problem is not “same port” by itself. The real rule is that only one API plugin can write to one parameter at a time in normal "set" mode. If plugin A controls MouthOpen and plugin B controls CustomMouthParam, that is fine. If both touch MouthOpen, VTS returns an error. (GitHub)

So for your one-port idea to work safely, you need all of these to be true:

  • terminal 1 only sends MouthOpen
  • terminal 2 only sends YourCustomParam
  • neither terminal ever writes the other parameter
  • your model or model-plus-item mapping is rigged so one character responds to one input and the other responds to the other input. (GitHub)

Would this cause more or less issues than what you have now?

For your current situation, I think it would usually cause more issues, not fewer. That is because you add another moving part: the custom parameter itself. Custom parameters must be created first, their names must be unique, and VTS will reject creation if a parameter with that name was already created by a different plugin. VTS also stores custom parameters in custom_parameters.json, and if a plugin’s authentication is revoked, the custom parameters created by that plugin are deleted. (GitHub)

So same-port plus custom mouth is valid, but it is more fragile than two cleanly separated instances. It adds:

  • socket sharing
  • plugin coordination
  • custom parameter lifecycle
  • rigging complexity. (GitHub)

About your token files

Using two token files is the correct direction. That reduces one major failure mode compared with a single shared token file. But it does not prove tokens are no longer part of the problem. The official API says you only need to obtain a token once and reuse it, and later authentication will fail unless pluginName and pluginDeveloper exactly match the values used when the token was created. Wrapper libraries reflect this by making persistent token storage a built-in feature. (GitHub)

So with two token files, the likely auth mistakes become:

  • terminal 1 reads terminal 2’s file by mistake
  • you changed pluginName
  • you changed pluginDeveloper
  • reconnect logic requests a fresh token instead of reusing the saved one
  • one terminal created the custom parameter, but later a different plugin identity tries to manage it. (GitHub)

That means your current issue could still be on your side, even with two token files. Two files reduce risk. They do not eliminate bad identity matching or bad reconnect logic. (GitHub)

The mouth behavior problem is probably not only tokens

There is another rule that matters a lot here: if you want VTS to keep respecting your plugin’s parameter value, you have to resend that parameter at least once every second. If you stop, the parameter is considered lost and falls back to whatever controlled it before, or to default. That often looks like “it worked for a bit and then broke.” (GitHub)

So if your setup works briefly, the cause could be:

  • auth mismatch on reconnect
  • wrong token file used
  • the terminal stops sending updates often enough
  • both terminals touch the same parameter
  • the custom parameter disappeared because the plugin permission was revoked. (GitHub)

My recommendation for your exact case

If you want the lowest-risk design, keep the two-instance layout:

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8002 → MouthOpen

That is simpler because each VTS instance already has its own MouthOpen, its own websocket port, and its own model context. (GitHub)

If you want the same-port design, do it only when you truly want one VTS instance. Then use:

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8001 → HyoriMouth or similar custom param

and keep the plugin identities stable, the token files separate, and the parameter ownership strict. That can work, but it is the more delicate setup. (GitHub)

My blunt answer is:

  • Possible: yes
  • Cleaner than two instances: no
  • Likely to reduce your current problems: probably no
  • Could the issue still be your implementation even with two token files: yes. (GitHub)

The most likely stable path is still two VTS instances, two ports, two token files, same standard MouthOpen, with each terminal tied to one port and one plugin identity. (GitHub)

Recommended default

Use two VTube Studio instances and keep the mouth parameter simple:

  • terminal 1 → VTS instance on 8001 → MouthOpen
  • terminal 2 → VTS instance on 8002 → MouthOpen

That matches how VTube Studio handles plugins. The API starts on 8001 by default, and if that port is already taken by another VTS instance, the next instance uses 8002, then higher. Each running VTS instance also has its own instanceID. (GitHub)

My reason for recommending this as the default is simple: it keeps instance routing, auth, and mouth control separated. Same-port designs are possible, but they add more moving parts. (GitHub)

Hard rules

1. Freeze plugin identity

For each terminal, pick one pluginName and one pluginDeveloper and do not change them casually. VTube Studio requires those values to match the ones used when the token was first requested, or authentication fails. (GitHub)

Good example:

  • terminal 1: Goober Mouth Driver / YourName
  • terminal 2: Hyori Mouth Driver / YourName

Bad example:

  • changing the plugin name every test run
  • using the same token with a different plugin name later (GitHub)

2. One terminal, one token file

You are already using two token files. Keep that. Store them separately and never let one terminal read or overwrite the other terminal’s file. Wrapper libraries are built around this exact idea: VTubeStudioJS exposes persistent token getter and setter hooks, and pyvts explicitly reads and writes a token file for reuse. (GitHub)

Good example:

tokens/
  goober_8001.token
  hyori_8002.token

3. Request the token once. Reuse it after that.

VTube Studio says you only need to obtain the token once. Later sessions should authenticate with the saved token instead of requesting a new one every time. Re-request only when authentication fails because the token is invalid or permissions were revoked. (GitHub)

4. Treat “same port” as “same VTS window”

If both terminals connect to 8001, they are both controlling the same VTube Studio instance. That is not a two-instance layout. If you want true separation between two VTubers, use 8001 and 8002. (GitHub)

5. One parameter, one owner

Only one plugin can set a given parameter at a time in normal "set" mode. If two plugins try to control the same parameter, VTS returns an error. This applies to both default and custom parameters. (GitHub)

That means:

  • safe: terminal 1 controls MouthOpen, terminal 2 controls HyoriMouth
  • unsafe: both terminals control MouthOpen in "set" mode (GitHub)

6. Keep the mouth alive

If you control a parameter through the API, VTS requires you to resend that parameter at least once every second. If you stop, the parameter is considered lost and control falls back to whatever was controlling it before, or to default. (GitHub)

For mouth animation, that means:

  • send updates continuously while speaking
  • stop only when you want control to fall back
  • do not assume “I sent one mouth-open value, so it will stay there” (GitHub)

7. Verify the loaded model before sending mouth values

Use CurrentModelRequest after connecting. It tells you whether a model is loaded and returns modelName and modelID. You can also subscribe to ModelLoadedEvent instead of polling. (GitHub)

Safe rule:

  • connect
  • authenticate
  • confirm the expected model is loaded
  • only then start sending mouth values (GitHub)

8. Do not add custom mouth parameters until auth is stable

Custom parameters are valid, but they are tied to plugin ownership. VTS stores them in custom_parameters.json, and if that plugin’s permission is revoked, the custom parameters created by that plugin are deleted. They remain referenced in models, but they turn red until recreated. (GitHub)

So custom parameters are fine when the system is stable. They are a bad first fix when auth and reconnect logic are still unreliable. (GitHub)

9. Check rig mapping before blaming the API

If the parameter appears to move but the mouth does not behave correctly, check the VTS model config. Expressions, animation, and physics can override visible motion. The model settings docs are the place to verify this. (GitHub)

10. If you use audio routing, isolate the input

VTube Studio’s voice lipsync uses the selected microphone input, not magic global sound detection. Its lipsync docs recommend keeping the VTS cutoff low and putting a noise gate before the audio reaches VTS. That means separate virtual cables can be clean if each VTS instance listens to its own dedicated input. (GitHub)

Safe layout for the two-instance design

This is the safest layout for you.

terminal 1
  pluginName: Goober Mouth Driver
  pluginDeveloper: YourName
  token file: tokens/goober_8001.token
  target port: 8001
  target model: WAFFLE LOVING GOOBER
  parameter: MouthOpen

terminal 2
  pluginName: Hyori Mouth Driver
  pluginDeveloper: YourName
  token file: tokens/hyori_8002.token
  target port: 8002
  target model: hyori
  parameter: MouthOpen

Startup flow:

  1. connect to assigned port
  2. authenticate with that terminal’s saved token
  3. if auth fails, request a new token and overwrite only that terminal’s token file
  4. call CurrentModelRequest
  5. confirm expected model
  6. stream MouthOpen while speaking
  7. on disconnect, reconnect to the same port and reuse the same token (GitHub)

Safe layout for the same-port custom-parameter design

Only use this if you intentionally want one VTS instance.

terminal 1 -> 8001 -> MouthOpen
terminal 2 -> 8001 -> HyoriMouth

Rules for this design:

  • terminal 1 never touches HyoriMouth
  • terminal 2 never touches MouthOpen
  • the custom parameter is created once and by a stable plugin identity
  • the model mapping is rigged so each character responds only to its intended input
  • both terminals still need their own token files and stable plugin identities (GitHub)

This works, but it is more fragile than the two-instance design because you are adding custom-parameter lifecycle and shared-instance coordination on top of the existing auth rules. (GitHub)

What your two token files do and do not prove

Using two token files is good. It removes one big failure mode: accidental sharing of one token file across two clients. Wrapper libraries strongly suggest this kind of persistent, per-client token storage. (GitHub)

But two files do not prove the issue is no longer auth-related. You can still break authentication by:

  • reading the wrong file
  • changing pluginName
  • changing pluginDeveloper
  • requesting a new token when you should reuse the old one (GitHub)

So with two token files already in place, the likely causes become:

  • identity mismatch
  • wrong port
  • wrong model loaded
  • two clients touching the same parameter
  • mouth updates not being resent often enough
  • custom parameter ownership or deletion if you went that route (GitHub)

Do / do not

Do

  • use 8001 and 8002 if you want two truly separate VTS windows (GitHub)
  • keep one stable plugin identity per terminal (GitHub)
  • keep one token file per terminal (GitHub)
  • verify the loaded model before starting mouth control (GitHub)
  • resend mouth values continuously while speaking (GitHub)

Do not

  • share a token file between terminals (GitHub)
  • change pluginName or pluginDeveloper after the token exists (GitHub)
  • let two terminals control the same parameter in "set" mode (GitHub)
  • switch to custom parameters before auth and reconnect logic are stable (GitHub)

My concrete recommendation

For your case, the safest practice is:

  1. stay with two VTS instances
  2. keep standard MouthOpen on each
  3. keep two token files
  4. freeze each terminal’s plugin identity
  5. verify the model after connect
  6. continuously resend mouth while that terminal’s TTS is speaking

I would only move to same-port plus custom mouth parameter if you explicitly want one VTS window and are willing to manage the extra fragility that comes with plugin-owned custom parameters. (GitHub)

6

Very long explanation but right now I’m thinking of making sure there is no possible for the token to switch to the other at any point in time.

Also Fixed a Connection issue where the api pop-up wasn’t working on terminal 1

Also wanna do the terminal 1 controls MouthOpen, terminal 2 controls HyoriMouth

And keep the token files separate and use different ports since it seems very common for it to be driven into errors otherwise for some reason

Right now i have a lot of conflicting code but I’ll fix it later but i do have a somewhat temporary solution for this although I’m still working on a fix

Also this:

Only use this if you intentionally want one VTS instance.

terminal 1 → 8001 → MouthOpen

terminal 2 → 8001 → HyoriMouth

Rules for this design:

terminal 1 never touches HyoriMouth

terminal 2 never touches MouthOpen

This also seems pretty safe to try personally later so I’ll test it later i guess As i could feed data of terminal 1 to only that openmouth and make terminal 2 only feed data into hyorimouth Despite using the same api port there would likely not being a issue with this setup

1 Like
7

Based on the software’s specifications, that should probably work.

Your current direction is sensible.

The strongest part is this: make token cross-use impossible, keep different ports, and remove as much “fallback” behavior as you can. VTube Studio expects tokens to be reused, but the token is tied to the pluginName and pluginDeveloper that created it. If those do not match later, auth fails. The official docs also show the normal pattern is persistent per-client token storage rather than ad hoc token swapping. (GitHub)

What I would keep

Keep this as your main plan for now:

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8002 → HyoriMouth
  • separate token files
  • separate plugin identities
  • no token fallback to the other file

Using different ports is the clean part here. VTube Studio starts on 8001 by default and moves to 8002, 8003, and so on for additional instances, so separate ports already isolate the two VTS windows. (GitHub)

Fixing the API popup issue on terminal 1 was also important. If the permission popup is not working, auth flow can fail before any mouth logic matters. The docs explicitly say the plugin requests permission through that popup, and users can revoke permission later from the plugin list. (GitHub)

One adjustment

With different ports, I would still consider starting with this first:

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8002 → MouthOpen

Then move terminal 2 to HyoriMouth only after the routing and auth are stable. The reason is simple: different ports already give you instance separation. HyoriMouth is not needed for port-level isolation. It is only useful if you want different rig behavior on Hyori’s side. Also, custom parameters are plugin-owned and get deleted if that plugin’s permission is revoked. (GitHub)

So HyoriMouth is a valid second step. I would not make it the first stabilization step unless you already know you need custom rig logic. (GitHub)

Your “same port later” idea

Yes. That later test is technically valid if you intentionally use one VTS instance and keep strict ownership:

  • terminal 1 → 8001 → MouthOpen
  • terminal 2 → 8001 → HyoriMouth

That works because VTS allows multiple connected plugins, and the restriction is per-parameter: only one plugin can control a given parameter in normal "set" mode, but different plugins can control different parameters. Custom parameters are explicitly supported. (GitHub)

So your logic is right here:

  • terminal 1 never touches HyoriMouth
  • terminal 2 never touches MouthOpen

If you obey that rule, the same-port design is reasonable to test later. The main warning is that same-port means same VTS instance, so now you are debugging shared-instance coordination and custom-parameter lifecycle in addition to auth. That is why it is a later experiment, not the best cleanup target while your codebase still has conflicting paths. (GitHub)

What I would change in your code first

Remove any logic that can do any of these:

  • try the other terminal’s token file
  • silently switch ports
  • silently switch parameters
  • request a fresh token before trying the saved one
  • let one terminal write both mouth params

Those are the exact kinds of mistakes that create “works on accident” behavior. The official docs are strict about stable token reuse, matching plugin identity, and one-plugin-per-parameter in "set" mode. (GitHub)

The safest order from here

  1. Freeze identities

    • terminal 1 has one plugin name, one token file, one port
    • terminal 2 has one plugin name, one token file, one port
      This matches the VTS auth model directly. (GitHub)

  2. Stabilize on separate ports

    • get 8001 and 8002 clean first
      That is the officially supported multi-instance pattern. (GitHub)

  3. Use plain MouthOpen first if possible

    • only switch terminal 2 to HyoriMouth if you still need it
      This avoids extra custom-parameter state while debugging. (GitHub)

  4. Only then test same-port later

    • one instance
    • different parameters
    • strict ownership
      This is valid, but it is a separate architecture choice. (GitHub)

Bottom line

Your current instincts are good.

  • Separate token files: keep that.
  • Different ports: keep that.
  • No possible token switching: absolutely do that.
  • Terminal 2 using HyoriMouth: valid, but better as a second step after the two-port setup is stable.
  • Same-port later with MouthOpen + HyoriMouth: safe enough to experiment with later, as long as it is intentionally a one-instance design and each terminal owns only its own parameter. (GitHub)

The shortest practical recommendation is: lock down token identity and ports first, stabilize there, then add HyoriMouth, then test same-port later if you still want it.