Langdrift

Agents that work perfectly in English often quietly degrade in other languages: same intent, different words, different behavior. Langdrift asks one question: does your agent still choose the right tool, with the right arguments, when the same request arrives in French, Arabic, Chinese, Basque, or Swahili? It checks behavior, not fluency; there is no LLM-as-judge in the core loop, so every failure is deterministic and explainable.

You can see the failure mode in 30 seconds without an API key. The repo ships a fake agent that intentionally drops tool calls for Swahili and Chinese:

Locale  Passed  Failure       Detail
en      1/1     -             create_refund_ticket
sw      0/1     no_tool_call  expected create_refund_ticket, got no tool calls
zh      0/1     no_tool_call  expected create_refund_ticket, got no tool calls

Result: failed, 2 of 12 locales failed

For your own agent, install the CLI (Node >= 24, zero runtime dependencies) and run it against your endpoint:

npm install -g langdrift
langdrift run ./my-scenario.yaml --target http://localhost:3010/api/agent

Scenarios are plain YAML. Argument assertions accept oneOf lists for canonical values, and responseLanguage verifies the reply’s script, useful for catching agents that fall back to English:

id: refund_request
agent: support

locales:
  en:
    input: "I was charged twice for my subscription"
    expect:
      toolCall:
        name: create_refund_ticket
        arguments:
          reason: duplicate_charge
  fr:
    input: "J'ai été facturé deux fois"
    expect:
      toolCall:
        name: create_refund_ticket
      responseLanguage: fr

This is not hypothetical. In the included benchmark, the same locale weaknesses recurred across three independently trained models: prompts in Basque, Yoruba, Swahili, Chinese, Welsh, and Mongolian triggered missing tool calls or wrong arguments where English passed. The full methodology, results, and limitations are in the repo’s RESEARCH.md.