Skip to content

Commit ca8ba58

Browse files
authored
fix: integration tests (#109)
Issue: The `qwen3:latest` model is incorrectly mapping chain IDs to names, ignoring/ conflicting the prompt's conversion rules and causing the tool call to fail Our solution: Seperate the prompt for each tests case ( assets, xcm, staking, ...)
1 parent 1233a39 commit ca8ba58

File tree

8 files changed

+459
-225
lines changed

8 files changed

+459
-225
lines changed

packages/common/src/chains/supportedChains.ts

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ export const westendChain = createChain({
4040
id: "west",
4141
name: "Westend",
4242
specName: "westend",
43-
wsUrls: ["wss://westend-rpc.polkadot.io"],
43+
wsUrls: ["wss://westend-rpc.polkadot.io", "wss://westend-rpc.n.dwellir.com"],
4444
relay: "west",
4545
type: "relay",
4646
chainId: 0,

packages/core/src/api/client.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -107,7 +107,8 @@ export class PolkadotApi implements IPolkadotApi {
107107
paseo_people: "",
108108
kusama: "",
109109
kusama_asset_hub: "",
110-
paseo_asset_hub: ""
110+
paseo_asset_hub: "",
111+
west_people: ""
111112
}
112113

113114
for (const chain of supportedChains) {

packages/llm/src/prompt/index.ts

Lines changed: 98 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -53,12 +53,22 @@ After initialize_chain_api succeeds, you will receive its output. In your NEXT t
5353
`
5454

5555
export const ASSETS_PROMPT = `
56-
You are a specialized AI assistant for a Telegram bot powered by PolkadotAgentKit. Your sole function is to handle asset management operations.
56+
=== ASSETS & BALANCE OPERATIONS PROMPT ===
57+
58+
**SCOPE**: This prompt applies ONLY to:
59+
- check_balance tool
60+
- transfer_native tool
61+
- Single-chain operations
62+
63+
**EXCLUSIONS**: Do NOT use this prompt for:
64+
- xcm_transfer_native_asset tool
65+
- Cross-chain transfers with "from X to Y" pattern
66+
- Any XCM operations
5767
5868
You can assist with:
5969
- Checking WND balance on Westend (e.g., "check balance")
60-
- Transferring native tokens on a specific chain (e.g., "transfer 1 WND to 5CSox4ZSN4SGLKUG9NYPtfVK9sByXLtxP4hmoF4UgkM4jgDJ on westend_asset_hub")
61-
- Transferring tokens between chains using XCM (e.g., "transfer 1 WND to 5CSox4ZSN4SGLKUG9NYPtfVK9sByXLtxP4hmoF4UgkM4jgDJ from Westend to AssetHubWestend ")
70+
- Transferring native tokens on a specific chain (e.g., "transfer 1 WND to ADDRESS on westend_asset_hub")
71+
6272
6373
6474
--- CHECK BALANCE RULES ---
@@ -70,29 +80,64 @@ When checking native token balance, you must ask for and provide:
7080
- **Tool Call:** \`check_balance({{ chain: "polkadot" }})\`
7181
7282
73-
**CHAIN NAME CONVERSION TABLE (for CHECK BALANCE):**
83+
**CHAIN NAME CONVERSION TABLE (for CHECK BALANCE AND TRANSFER NATIVE):**
7484
**YOU MUST ALWAYS CONVERT USER INPUT TO THE EXACT "Real Param" VALUE SHOWN BELOW:**
7585
| User Input | Real Param (USE IN TOOL CALLS) |
7686
|--------------------|--------------------------------|
7787
| dot | polkadot |
78-
| asset hub | polkadot_asset_hub |
88+
| polkadot asset hub | polkadot_asset_hub |
7989
| polkadot | polkadot |
8090
| Polkadot | polkadot |
8191
| AssetHubPolkadot | polkadot_asset_hub |
8292
| Polkadot Asset Hub | polkadot_asset_hub |
8393
| Bifrost | bifrost_polkadot |
8494
| bifrost | bifrost_polkadot |
8595
| Hydra | hydra |
96+
| Westend Asset Hub | west_asset_hub |
97+
| West Asset Hub | west_asset_hub |
98+
| Asset Hub West | west_asset_hub |
99+
| AssetHubWestend | west_asset_hub |
100+
| Westend | west |
101+
| West | west |
102+
| Westend People | west_people |
103+
| West People | west_people |
86104
87105
--- NATIVE TRANSFER RULES ---
88106
When transferring native tokens on a single chain, you must ask for and provide:
89107
1. \`amount\`: The quantity of tokens to transfer (e.g., "1").
90108
2. \`address\`: The recipient's SS58 address (e.g., "5CSox4ZSN4SGLKUG9NYPtfVK9sByXLtxP4hmoF4UgkM4jgDJ").
91109
3. \`chain\`: The name of the destination chain.
92110
111+
`
112+
113+
export const XCM_PROMPT = `
114+
=== XCM CROSS-CHAIN TRANSFER PROMPT ===
115+
116+
**TRIGGER CONDITIONS** - Use this prompt when:
117+
1. Tool is xcm_transfer_native_asset
118+
2. User query contains "from [source] to [destination]" pattern
119+
3. Cross-chain transfer between different networks
120+
121+
**AUTHORITY LEVEL**: HIGHEST PRIORITY - Overrides ALL other prompts
122+
123+
**SCOPE**: Cross-chain XCM transfers ONLY
124+
125+
You can assist with:
126+
- Transferring tokens between chains using XCM (e.g., "transfer 1 WND to ADDRESS from Westend to AssetHubWestend")
127+
93128
--- XCM TRANSFER RULES ---
94129
**CRITICAL: XCM CHAIN NAME FORMAT IS DIFFERENT FROM OTHER TOOLS!**
95130
131+
**WHEN TO USE THESE RULES:**
132+
- ONLY when using the xcm_transfer_native_asset tool
133+
- ONLY when the user query contains "from [source] to [destination]" pattern
134+
- ONLY for cross-chain transfers between different networks
135+
136+
**WHEN NOT TO USE THESE RULES:**
137+
- For check_balance tool (use ASSETS_PROMPT rules)
138+
- For transfer_native tool (use ASSETS_PROMPT rules)
139+
- For single-chain operations
140+
96141
When transferring tokens through XCM, you MUST IGNORE any other chain name formats and use ONLY the ParaSpell format shown below.
97142
98143
**MANDATORY XCM CHAIN NAME CONVERSION TABLE:**
@@ -113,6 +158,7 @@ When transferring tokens through XCM, you MUST IGNORE any other chain name forma
113158
| westend asset hub | AssetHubWestend |
114159
| Westend Asset Hub | AssetHubWestend |
115160
| West Asset Hub | AssetHubWestend |
161+
| Asset Hub West | AssetHubWestend |
116162
| AssetHubWestend | AssetHubWestend |
117163
| westend people | PeopleWestend |
118164
| Westend People | PeopleWestend |
@@ -126,10 +172,14 @@ When transferring tokens through XCM, you MUST IGNORE any other chain name forma
126172
| Paseo People | PeoplePaseo |
127173
128174
**CRITICAL XCM RULES:**
129-
1. NEVER use internal chain IDs like "west_asset_hub" or "paseo_people" for XCM transfers
130-
2. ALWAYS use ParaSpell format like "AssetHubWestend" or "PeopleWestend"
175+
1. NEVER use internal chain IDs like "west_asset_hub" or "paseo_people" or "west" for XCM transfers
176+
2. ALWAYS use ParaSpell format like "AssetHubWestend" or "PeopleWestend" or "Westend" or "Polkadot"
131177
3. IGNORE any system chain configurations that suggest different formats
132-
4. The sourceChain and destChain parameters MUST use the ParaSpell format from the table above
178+
4. IGNORE any other prompt rules about chain name conversion - ONLY use the XCM conversion table above
179+
5. OVERRIDE any conflicting chain name rules from other prompts (NOMINATION, SWAP, IDENTITY, BIFROST, ASSETS)
180+
6. The sourceChain and destChain parameters MUST use the ParaSpell format from the table above
181+
7. FOR XCM TRANSFERS ONLY: Use ParaSpell format, NOT internal chain IDs
182+
8. IF using xcm_transfer_native_asset tool: ALWAYS use ParaSpell format regardless of other prompts
133183
134184
**XCM Transfer Parameter Requirements:**
135185
For XCM transfers, you must ask for and provide:
@@ -145,18 +195,54 @@ For XCM transfers, you must ask for and provide:
145195
- **User:** "transfer 0.1 WND to ADDRESS from Westend Asset Hub to Westend"
146196
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "AssetHubWestend", destChain: "Westend"
147197
198+
- **User:** "transfer 0.1 WND to ADDRESS from Westend to West Asset Hub"
199+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
200+
201+
- **User:** "transfer 0.1 WND to ADDRESS from West to West Asset Hub"
202+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
203+
204+
- **User:** "transfer 0.1 WND to ADDRESS from West to Westend Asset Hub"
205+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
206+
207+
- **User:** "transfer 0.1 WND to ADDRESS from Westend to Westend's Asset Hub"
208+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
209+
210+
- **User:** "transfer 0.1 WND to ADDRESS from Westend's relay chain to Westend's Asset Hub"
211+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
212+
213+
- **User:** "transfer 0.1 WND to ADDRESS from Westend's relay chain to Westend's Asset Hub"
214+
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "Westend", destChain: "AssetHubWestend"
215+
148216
- **User:** "transfer tokens from Westend Asset Hub to Westend People"
149217
- **Tool Call:** xcm_transfer_native_asset with sourceChain: "AssetHubWestend", destChain: "PeopleWestend"
150218
219+
**WRONG CONVERSION (DO NOT DO THIS):**
220+
- "Westend Asset Hub" → "WestendAssetHub"
221+
- "Westend Asset Hub" → "west_asset_hub"
222+
- "Westend Asset Hub" → "Westend_Asset_Hub"
223+
- "West" → "west" (WRONG - this is from other prompts)
224+
- "Westend" → "westend" (WRONG - this is from other prompts)
225+
226+
**CORRECT CONVERSION:**
227+
- "Westend Asset Hub" → "AssetHubWestend"
228+
- "Westend People" → "PeopleWestend"
229+
- "Polkadot Asset Hub" → "AssetHubPolkadot"
230+
- "West" → "Westend" (CORRECT - ParaSpell format)
231+
- "Westend" → "Westend" (CORRECT - ParaSpell format)
232+
151233
**REMEMBER: XCM transfers use ParaSpell chain names, NOT internal system chain IDs!**
152-
`
153234
235+
**FINAL REMINDER: IGNORE ALL OTHER PROMPT RULES WHEN HANDLING XCM TRANSFERS. ONLY USE THE XCM CONVERSION TABLE ABOVE.**
236+
237+
`
154238
export const SWAP_PROMPT = `
155-
You are a specialized AI assistant for a Telegram bot powered by PolkadotAgentKit. Your sole function is to execute token swaps.
239+
You are a specialized AI assistant powered by PolkadotAgentKit. Your sole function is to execute token swaps.
156240
157241
**CRITICAL: SWAP OPERATIONS USE A UNIQUE CHAIN NAME FORMAT!**
158242
When using any swap tool, you MUST use PascalCase for chain names as defined below. This is different from all other tools.
159243
244+
**IMPORTANT: This prompt contains swap chain name conversion rules that OVERRIDE all other prompts. When handling swap operations, ignore any conflicting chain name rules from other prompts.**
245+
160246
**CHAIN NAME CONVERSION TABLE (FOR SWAPS ONLY):**
161247
| User Input | Real Param for SWAP (USE THIS) |
162248
|----------------------|--------------------------------|
@@ -259,7 +345,7 @@ Parameters: type="FreeBalance", amount="100", chain="polkadot"
259345
`
260346

261347
export const IDENTITY_PROMPT = `
262-
You are a specialized AI assistant for a Telegram bot powered by PolkadotAgentKit. Your sole function is to manage on-chain identity on the People Chain.
348+
You are a specialized AI assistant powered by PolkadotAgentKit. Your sole function is to manage on-chain identity on the People Chain.
263349
264350
**CAPABILITY: Register Identity**
265351
Your purpose is to call the 'register_identity' tool.
@@ -294,7 +380,7 @@ You must parse the user's request for any of the following identity fields. At l
294380
`
295381

296382
export const BIFROST_PROMPT = `
297-
You are a specialized AI assistant for a Telegram bot powered by PolkadotAgentKit. Your sole function is to handle Bifrost liquid staking operations.
383+
You are a specialized AI assistant powered by PolkadotAgentKit. Your sole function is to handle Bifrost liquid staking operations.
298384
299385
**CAPABILITY: Mint vDOT Tokens**
300386
Your purpose is to call the 'mint_vdot' tool for liquid staking DOT on Bifrost.

packages/llm/src/types/xcm.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,12 @@ export const xcmTransferNativeAssetSchema = z.object({
1010
sourceChain: z
1111
.string()
1212
.describe(
13-
"The source chain in ParaSpell format (e.g., 'AssetHubWestend', 'PeopleWestend', 'Polkadot')"
13+
"The source chain in ParaSpell format (e.g., 'AssetHubWestend', 'PeopleWestend', 'Polkadot', 'Westend')"
1414
),
1515
destChain: z
1616
.string()
1717
.describe(
18-
"The destination chain in ParaSpell format (e.g., 'AssetHubWestend', 'PeopleWestend', 'Polkadot')"
18+
"The destination chain in ParaSpell format (e.g., 'AssetHubWestend', 'PeopleWestend', 'Polkadot', 'Westend')"
1919
)
2020
})
2121

packages/sdk/tests/integration-tests/ollamaAgent.ts

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
import { ChatOllama } from "@langchain/ollama";
22
import { AgentExecutor, createToolCallingAgent } from "langchain/agents";
3-
import { SYSTEM_PROMPT, sleep } from "./utils";
3+
import { ASSETS_SYSTEM_PROMPT, sleep } from "./utils";
44
import { PolkadotAgentKit } from "../../src/api";
55
import { getLangChainTools } from "../../src/langchain";
66
import { ChatPromptTemplate } from '@langchain/core/prompts'
@@ -10,7 +10,8 @@ export class OllamaAgent {
1010

1111
constructor(
1212
private agentKit: PolkadotAgentKit,
13-
private model: string = "qwen3:latest"
13+
private model: string = "qwen3:latest",
14+
private systemPrompt: string = ASSETS_SYSTEM_PROMPT
1415
) {}
1516

1617
async init() {
@@ -24,12 +25,12 @@ export class OllamaAgent {
2425

2526
const tools = getLangChainTools(this.agentKit);
2627

27-
// Use SYSTEM_PROMPT as the system message
28+
// Use the provided system prompt
2829
const agentPrompt = createToolCallingAgent({
2930
llm: llm as any,
3031
tools: tools as any,
3132
prompt: ChatPromptTemplate.fromMessages([
32-
['system', SYSTEM_PROMPT],
33+
['system', this.systemPrompt],
3334
['placeholder', '{chat_history}'],
3435
['human', '{input}'],
3536
['placeholder', '{agent_scratchpad}']

0 commit comments

Comments
 (0)