Skip to content

Commit 79b80c6

Browse files
committed
- Added experimental support for Chatterbox and Chatterbox Turbo text-to-speech (requires a specific configuration - will need to write a guide).
- Renamed the "AI Reminders" button to "Problems?" and changed its icon, to reduce confusion. - Added a new "Wrong Speech Styles" injection prompt useful in case the model is confusing speech patterns, such as when one character is speaking the same way as someone else. - Added the Asset Quality option in the Settings menu, allowing users to switch between high quality and low quality models, with a noticeable performance improvement, especially on low-end hardware, for selecting the latter option. - More improvements to the Pollinations API provider: removed free models that do not work properly and automatically refresh the model list when the user is inputting an API key or is removing it.
1 parent 5589538 commit 79b80c6

File tree

4 files changed

+267
-15
lines changed

4 files changed

+267
-15
lines changed

src/components/views/ChatInterface.vue

Lines changed: 101 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -87,18 +87,19 @@
8787
:style="{ marginLeft: '4px', opacity: isLoading ? 0.4 : 0.8, transition: 'opacity 0.15s' }"
8888
>
8989
<template #icon>
90-
<n-icon><Notification /></n-icon>
90+
<n-icon><Help /></n-icon>
9191
</template>
92-
AI Reminders
92+
Problems?
9393
</n-button>
9494
</template>
9595
<div style="display: flex; flex-direction: column; gap: 8px; padding: 4px; min-width: 150px;">
9696
<div style="font-weight: 600; font-size: 12px; opacity: 0.7; border-bottom: 1px solid var(--n-border-color); padding-bottom: 4px; margin-bottom: 2px;">
97-
Active Reminders
97+
Inject one or more reminders to the model for the next turn:
9898
</div>
9999
<n-checkbox v-model:checked="invalidJsonToggle">Invalid JSON Schema</n-checkbox>
100100
<n-checkbox v-model:checked="honorificsToggle">Incorrect Honorifics</n-checkbox>
101101
<n-checkbox v-model:checked="narrationAndDialogueNotSplitToggle">Narration and Dialogue Not Split</n-checkbox>
102+
<n-checkbox v-model:checked="wrongSpeechStylesToggle">Using Wrong Speech Styles</n-checkbox>
102103
<n-checkbox v-if="mode === 'roleplay'" v-model:checked="aiControllingUserToggle">AI Is Controlling Me</n-checkbox>
103104
</div>
104105
</n-popover>
@@ -267,6 +268,23 @@
267268

268269
<h4 class="settings-section-header accent-red">Interface & Audio</h4>
269270
<n-divider />
271+
<n-form-item>
272+
<template #label>
273+
Asset Quality
274+
<n-popover trigger="hover" placement="bottom">
275+
<template #trigger>
276+
<n-icon size="16" style="vertical-align: text-bottom; margin-left: 4px; cursor: help; color: #888">
277+
<Help />
278+
</n-icon>
279+
</template>
280+
<div>
281+
<strong>High:</strong> Best visual quality, but uses more GPU resources and VRAM.<br><br>
282+
<strong>Low:</strong> Slightly worse quality and detail, but much better performance on low-end systems.
283+
</div>
284+
</n-popover>
285+
</template>
286+
<n-select v-model:value="assetQuality" :options="assetQualityOptions" />
287+
</n-form-item>
270288
<n-form-item>
271289
<template #label>
272290
Playback
@@ -350,6 +368,29 @@
350368
</template>
351369
<n-input v-model:value="gptSovitsBasePath" placeholder="C:/GPT-SoVITS" />
352370
</n-form-item>
371+
372+
<n-form-item label="Chatterbox Endpoint" v-if="ttsEnabled && ttsProvider === 'chatterbox'">
373+
<n-input v-model:value="chatterboxEndpoint" placeholder="http://localhost:7860" />
374+
</n-form-item>
375+
376+
<n-form-item v-if="ttsEnabled && ttsProvider === 'chatterbox'">
377+
<template #label>
378+
Chatterbox Voices Path
379+
<n-popover trigger="hover" placement="bottom">
380+
<template #trigger>
381+
<n-icon size="16" style="vertical-align: text-bottom; margin-left: 4px; cursor: help; color: #888">
382+
<Help />
383+
</n-icon>
384+
</template>
385+
<div>
386+
Path to your Chatterbox voices folder.<br>
387+
Voice files should be placed at: <code>{voicesPath}/{character}.wav</code><br>
388+
Example: <code>anis.wav</code>, <code>neon.wav</code>, etc.
389+
</div>
390+
</n-popover>
391+
</template>
392+
<n-input v-model:value="chatterboxVoicesPath" placeholder="C:/chatterbox-tts-api/voices" />
393+
</n-form-item>
353394
<n-divider />
354395
</n-form>
355396
</n-drawer-content>
@@ -436,7 +477,7 @@ import { marked } from 'marked'
436477
import { animationMappings } from '@/utils/animationMappings'
437478
import { cleanWikiContent, sanitizeActions, splitNarration, parseFallback, parseAIResponse, isWholeWordPresent } from '@/utils/chatUtils'
438479
import { normalizeAiActionCharacterData } from '@/utils/aiActionNormalization'
439-
import { ttsEnabled, ttsEndpoint, ttsProvider, gptSovitsEndpoint, gptSovitsBasePath, ttsProviderOptions, playTTS } from '@/utils/ttsUtils'
480+
import { ttsEnabled, ttsEndpoint, ttsProvider, gptSovitsEndpoint, gptSovitsBasePath, chatterboxEndpoint, chatterboxVoicesPath, ttsProviderOptions, playTTS } from '@/utils/ttsUtils'
440481
import { allowWebSearchFallback, NATIVE_SEARCH_PREFIXES, POLLINATIONS_NATIVE_SEARCH_MODELS, hasNativeSearch, usesWikiFetch, usesPollinationsAutoFallback, webSearchFallbackHelpText, searchForCharacters, searchForCharactersPerplexity, searchForCharactersWithNativeSearch, searchForCharactersViaWikiFetch } from '@/utils/aiWebSearchUtils'
441482
import { callOpenRouterSummarization, callPollinationsSummarization, callGeminiSummarization } from '@/utils/llmUtils'
442483
@@ -563,6 +604,7 @@ const invalidJsonToggle = ref(false)
563604
const honorificsToggle = ref(false)
564605
const aiControllingUserToggle = ref(false)
565606
const narrationAndDialogueNotSplitToggle = ref(false)
607+
const wrongSpeechStylesToggle = ref(false)
566608
567609
// Helper to set random loading message
568610
const setRandomLoadingMessage = () => {
@@ -609,6 +651,11 @@ const playbackOptions = [
609651
{ label: 'Manual', value: 'manual' }
610652
]
611653
654+
const assetQualityOptions = [
655+
{ label: 'High', value: 'high' },
656+
{ label: 'Low', value: 'low' }
657+
]
658+
612659
const tokenUsageOptions = [
613660
{ label: 'Low (10 turns)', value: 'low' },
614661
{ label: 'Medium (30 turns)', value: 'medium' },
@@ -625,9 +672,22 @@ const computedUsesPollinationsAutoFallback = computed(() => usesPollinationsAuto
625672
626673
const computedWebSearchFallbackHelpText = computed(() => webSearchFallbackHelpText(computedUsesWikiFetch.value, computedUsesPollinationsAutoFallback.value))
627674
675+
const assetQuality = computed({
676+
get: () => market.live2d.HQassets ? 'high' : 'low',
677+
set: (val: string) => {
678+
market.live2d.HQassets = val === 'high'
679+
}
680+
})
681+
628682
// Watchers
629-
watch(apiKey, (newVal) => {
683+
watch(apiKey, async (newVal) => {
630684
localStorage.setItem('nikke_api_key', newVal)
685+
if (apiProvider.value === 'pollinations') {
686+
await fetchPollinationsModels()
687+
if (pollinationsModels.value.length > 0) {
688+
model.value = pollinationsModels.value[0].value
689+
}
690+
}
631691
})
632692
633693
watch(useLocalProfiles, (newVal) => {
@@ -686,6 +746,14 @@ watch(gptSovitsBasePath, (newVal) => {
686746
localStorage.setItem('nikke_gptsovits_basepath', newVal)
687747
})
688748
749+
watch(chatterboxEndpoint, (newVal) => {
750+
localStorage.setItem('nikke_chatterbox_endpoint', newVal)
751+
})
752+
753+
watch(chatterboxVoicesPath, (newVal) => {
754+
localStorage.setItem('nikke_chatterbox_voices_path', newVal)
755+
})
756+
689757
watch(godModeEnabled, (newVal) => {
690758
localStorage.setItem('nikke_god_mode_enabled', String(newVal))
691759
})
@@ -698,6 +766,11 @@ watch(() => market.live2d.yapEnabled, (newVal) => {
698766
localStorage.setItem('nikke_yap_enabled', String(newVal))
699767
})
700768
769+
watch(() => market.live2d.HQassets, (newVal) => {
770+
localStorage.setItem('nikke_hq_assets', String(newVal))
771+
market.live2d.triggerResetPlacement()
772+
})
773+
701774
watch(apiProvider, async (newVal) => {
702775
localStorage.setItem('nikke_api_provider', newVal)
703776
@@ -773,12 +846,13 @@ const fetchPollinationsModels = async () => {
773846
774847
pollinationsModels.value = models
775848
.filter((m: any) => {
776-
// Hide certain models when using API key
777-
if (apiKey.value) {
849+
if (!apiKey.value) {
850+
const allowedModels = ['gemini', 'gemini-search', 'mistral']
851+
return allowedModels.includes(m.name)
852+
} else {
778853
const hiddenModels = ['qwen-coder', 'chickytutor', 'midijourney', 'openai-audio']
779854
return !hiddenModels.includes(m.name)
780855
}
781-
return true
782856
})
783857
.map((m: any) => {
784858
const isFree = m.pricing && m.pricing.input_token_price === 0
@@ -800,9 +874,9 @@ const fetchPollinationsModels = async () => {
800874
console.error('Failed to fetch Pollinations models:', error)
801875
// Fallback to hardcoded models
802876
models = [
803-
{ name: 'openai', pricing: { input_token_price: 0 } },
804-
{ name: 'mistral', pricing: { input_token_price: 0 } },
805-
{ name: 'searchgpt', pricing: { input_token_price: 0 } }
877+
{ name: 'gemini', pricing: { input_token_price: 0 } },
878+
{ name: 'gemini-search', pricing: { input_token_price: 0 } },
879+
{ name: 'mistral', pricing: { input_token_price: 0 } }
806880
]
807881
808882
pollinationsModels.value = models.map((m: any) => ({
@@ -840,21 +914,30 @@ const initializeSettings = async () => {
840914
const savedYap = localStorage.getItem('nikke_yap_enabled')
841915
if (savedYap !== null) market.live2d.yapEnabled = (savedYap === 'true')
842916
917+
const savedHQAssets = localStorage.getItem('nikke_hq_assets')
918+
if (savedHQAssets !== null) market.live2d.HQassets = (savedHQAssets === 'true')
919+
843920
const savedTts = localStorage.getItem('nikke_tts_enabled')
844921
if (savedTts !== null) ttsEnabled.value = (savedTts === 'true')
845922
846923
const savedTtsEndpoint = localStorage.getItem('nikke_tts_endpoint')
847924
if (savedTtsEndpoint) ttsEndpoint.value = savedTtsEndpoint
848925
849926
const savedTtsProvider = localStorage.getItem('nikke_tts_provider')
850-
if (savedTtsProvider === 'alltalk' || savedTtsProvider === 'gptsovits') ttsProvider.value = savedTtsProvider
927+
if (savedTtsProvider === 'alltalk' || savedTtsProvider === 'gptsovits' || savedTtsProvider === 'chatterbox') ttsProvider.value = savedTtsProvider
851928
852929
const savedGptSovitsEndpoint = localStorage.getItem('nikke_gptsovits_endpoint')
853930
if (savedGptSovitsEndpoint) gptSovitsEndpoint.value = savedGptSovitsEndpoint
854931
855932
const savedGptSovitsBasePath = localStorage.getItem('nikke_gptsovits_basepath')
856933
if (savedGptSovitsBasePath) gptSovitsBasePath.value = savedGptSovitsBasePath
857934
935+
const savedChatterboxEndpoint = localStorage.getItem('nikke_chatterbox_endpoint')
936+
if (savedChatterboxEndpoint) chatterboxEndpoint.value = savedChatterboxEndpoint
937+
938+
const savedChatterboxVoicesPath = localStorage.getItem('nikke_chatterbox_voices_path')
939+
if (savedChatterboxVoicesPath) chatterboxVoicesPath.value = savedChatterboxVoicesPath
940+
858941
const savedTokenUsage = localStorage.getItem('nikke_token_usage')
859942
if (savedTokenUsage && tokenUsageOptions.some((t) => t.value === savedTokenUsage)) tokenUsage.value = savedTokenUsage
860943
@@ -1209,6 +1292,7 @@ const sendMessage = async () => {
12091292
honorificsToggle.value = false
12101293
narrationAndDialogueNotSplitToggle.value = false
12111294
aiControllingUserToggle.value = false
1295+
wrongSpeechStylesToggle.value = false
12121296
}
12131297
12141298
isLoading.value = false
@@ -1270,6 +1354,7 @@ const retryLastMessage = async () => {
12701354
honorificsToggle.value = false
12711355
narrationAndDialogueNotSplitToggle.value = false
12721356
aiControllingUserToggle.value = false
1357+
wrongSpeechStylesToggle.value = false
12731358
}
12741359
12751360
isLoading.value = false
@@ -1367,6 +1452,7 @@ const continueStory = async () => {
13671452
honorificsToggle.value = false
13681453
narrationAndDialogueNotSplitToggle.value = false
13691454
aiControllingUserToggle.value = false
1455+
wrongSpeechStylesToggle.value = false
13701456
}
13711457
13721458
isLoading.value = false
@@ -1389,6 +1475,9 @@ const injectUserReminders = (messages: any[]): any[] => {
13891475
if (aiControllingUserToggle.value) {
13901476
reminders += '\n\n' + prompts.reminders.aiControllingUserReminder
13911477
}
1478+
if (wrongSpeechStylesToggle.value) {
1479+
reminders += '\n\n' + prompts.reminders.wrongSpeechStylesReminder
1480+
}
13921481
13931482
if (!reminders) return messages
13941483

src/utils/json/prompts.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,7 +41,8 @@
4141
"honorificsReminder": "[CRITICAL INSTRUCTIONS: You have used honorifics wrong in your last message. Remember that honorifics should only be used towards the Commander ONLY. Pay attention to the table to understand who should use which honorific.]",
4242
"speechStyleReminder": "[CRITICAL INSTRUCTIONS: Your last message did not reflect the characters speech styles correctly. Please ensure that each character's dialogue matches their unique speech patterns, tone, and vocabulary as specified in their speech_style section of their profile.]",
4343
"aiControllingUserReminder": "[CRITICAL INSTRUCTIONS: In the last messages, you were controlling the Commander's actions or speaking in his place. In Roleplay Mode, the AI should NOT control what the Commander (the user) is doing in the story or speak in his place, unless it is something critical for the advancement of the plot (e.g. The Commander's squad is entering a building and the user does not react with any action to that).]",
44-
"narrationAndDialogueNotSplit": "[CRITICAL INSTRUCTIONS: Your last message did not properly separate dialogue from narration. Remember to use separate actions in the JSON array.]"
44+
"narrationAndDialogueNotSplit": "[CRITICAL INSTRUCTIONS: Your last message did not properly separate dialogue from narration. Remember to use separate actions in the JSON array.]",
45+
"wrongSpeechStylesReminder": "[CRITICAL INSTRUCTIONS: You are confusing the characters' speech styles. Please ensure that each character is using its own, correct speech style according to each characterProfiles key.]"
4546
},
4647
"continue": {
4748
"story": "Continue the story with multiple dialogue exchanges and/or actions. Do not repeat previous events, dialogue or actions.",

0 commit comments

Comments
 (0)