BoxLang AI v2.1.0 Released

BoxLang AI v2.1.0 is here with powerful enterprise capabilities that make building production AI applications easier than ever. This release delivers multi-tenant usage tracking, OpenSearch vector memory, full AWS Bedrock support, Docker Desktop AI integration, and streamlined provider configuration.

🚀 Highlights

Complete release notes and migration guides are available at https://ai.ortusbooks.com/readme/release-history/2.1.0. Our documentation includes an integrated MCP server for agentic IDE support: https://ai.ortusbooks.com/~gitbook/mcp

Docker Desktop AI Integration

Develop and test AI features locally with Docker Desktop models—no internet required, no API costs, complete privacy.

// Automatic Docker Desktop model detection
result = aiChat(
    provider: "docker",
    messages: "Explain BoxLang closures",
    params: {
        model: "llama3.2:latest"
    }
)

// Stream from local models
aiChatStream(
    provider: "docker",
    messages: "Write a BoxLang function",
    params: {
        model: "qwen2.5:0.5b-instruct"
    },
    onChunk: ( chunk ) => {
        writeOutput( chunk.content )
        flush()
    }
)

Ideal for:

• Offline development when connectivity is limited
• Zero-cost prototyping without API charges
• Private deployments requiring data isolation
• Fast iteration with immediate model availability

Provider-Agnostic Usage Tracking

Enterprise-grade usage tracking with tenant-level billing support across all AI providers.

// Add tenant context to any request
result = aiChat(
    messages: "Generate quarterly sales report",
    options: {
        tenantId: "customer-123",
        usageMetadata: {
            costCenter: "sales",
            projectId: "q4-2024",
            userId: "john@company.com"
        }
    }
)

Real-time usage monitoring with interceptors:

BoxRegisterInterceptor( {
    class: "BillingInterceptor",
    points: {
        onAITokenCount: ( event, data ) => {
            // data.usage, data.tenantId, data.usageMetadata
            queryExecute(
                "INSERT INTO usage_logs (tenant_id, tokens, metadata, timestamp) 
                 VALUES (:tid, :tokens, :meta, :ts)",
                {
                    tid: data.tenantId,
                    tokens: data.usage.total_tokens,
                    meta: serializeJSON( data.usageMetadata ),
                    ts: now()
                }
            )
        }
    }
} )

Works with OpenAI, Bedrock, Ollama, Anthropic, Gemini, DeepSeek, and all supported providers.

Enterprise Vector Search with OpenSearch

Scale to millions of conversation messages with AWS OpenSearch or self-hosted clusters. Full semantic search capabilities for production RAG applications.

// AWS OpenSearch with SigV4 auth
memory = aiMemory(
    provider: "opensearch",
    options: {
        url: "https://search-domain.us-east-1.es.amazonaws.com",
        indexName: "conversations",
        embeddingProvider: "openai",
        vectorDimension: 1536,
        awsRegion: "us-east-1"
    }
)

// Built-in multi-tenant isolation
memory.add(
    messages: messages,
    userId: "user-123",
    conversationId: "support-ticket-456"
)

results = memory.query(
    query: "pricing discussions",
    userId: "user-123",
    limit: 5
)

Fine-tune HNSW algorithm parameters:

memory = aiMemory(
    provider: "opensearch",
    options: {
        spaceType: "cosinesimilarity",
        hnswM: 16,
        hnswEfConstruction: 100,
        hnswEfSearch: 100
    }
)

Centralized Provider Configuration

Configure once, use everywhere. Define provider defaults in module settings for cleaner, more maintainable code.

// ModuleConfig.bx
settings = {
    providers: {
        "openai": {
            params: { model: "gpt-4" },
            options: { apiKey: getSystemSetting( "OPENAI_KEY" ) }
        },
        "ollama": {
            params: { model: "qwen2.5:0.5b-instruct" },
            options: { baseUrl: "http://ollama.internal:11434/" }
        }
    }
}

Clean usage throughout your application:

// Uses configured defaults automatically
result = aiChat( provider: "openai", messages: "Hello" )

// Override defaults when needed
model = aiModel( 
    provider: "ollama",
    options: { logRequestToConsole: true }
)

Complete AWS Bedrock Integration

Full-featured Bedrock support with chat, streaming, and embeddings across all model families: Claude, Titan, Llama, Mistral.

// Chat completions
result = aiChat(
    provider: "bedrock",
    messages: "Analyze customer sentiment",
    params: { 
        model: "anthropic.claude-3-sonnet-20240229-v1:0" 
    }
)

// Real-time streaming
aiChatStream(
    provider: "bedrock",
    messages: "Write a detailed product description",
    params: { 
        model: "anthropic.claude-3-sonnet-20240229-v1:0" 
    },
    onChunk: ( chunk ) => {
        writeOutput( chunk.content )
        flush()
    }
)

// Vector embeddings
embeddings = aiEmbeddings(
    provider: "bedrock",
    input: "Customer feedback analysis",
    params: {
        model: "amazon.titan-embed-text-v1"
    }
)

Leverage inference profiles for optimized performance:

result = aiChat(
    provider: "bedrock",
    messages: "Analyze customer sentiment",
    options: {
        providerOptions: {
            inferenceProfileArn: "arn:aws:bedrock:us-east-1:123:inference-profile/production"
        }
    }
)

OpenAI-Compatible Embedding Endpoints

Use any OpenAI-compatible embedding service with vector memory providers—Ollama, LM Studio, or custom endpoints.

// Ollama embeddings
memory = aiMemory(
    provider: "pinecone",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:11434/v1"
        }
    }
)

// LM Studio embeddings
memory = aiMemory(
    provider: "qdrant",
    options: {
        embeddingProvider: "openai",
        embeddingOptions: {
            baseURL: "http://localhost:1234/v1"
        }
    }
)

🔧 Additional Enhancements

• New Event: onMissingAiProvider for graceful provider fallback handling
• Enhanced Configuration: aiModel() accepts options struct for flexible service initialization
• Request Control: mergeServiceParams() and mergeServiceHeaders() now support override argument
• Ollama Embeddings: Added nomic-embed-text model support for local embedding generation
• Event Naming: Corrected to onAIChatRequest, onAIChatRequestCreate, onAIChatResponse
• Header Support: Fixed header passthrough in aiChat() and aiChatStream()
• MCP Standards: Prompts return arguments key per MCP specification
• Model Access: Fixed AiModel.getModel() with predefined providers
• Docker Reliability: Increased Model Runner retry timing for large model initialization

📚 Migration Path

Fully backward compatible—all new features are opt-in:

1. Enable usage tracking – Add tenantId and usageMetadata to track per-tenant costs
2. Centralize configuration – Move provider settings to module config
3. Scale with OpenSearch – Deploy production vector search infrastructure
4. Integrate Bedrock – Enable chat, streaming, and embeddings on AWS
5. Customize embeddings – Point to self-hosted or custom embedding services

🚀 Installation

Install via our package managers: https://ai.ortusbooks.com/getting-started/installation

// BoxLang CLI
install-bx-module bx-ai

// CommandBox
box install bx-ai

Resources

• Full Documentation: ai.ortusbooks.com
• Project Site: ai.boxlang.io
• AI Bootcamp: github.com/ortus-boxlang/bx-ai/bootcamp
• Code Examples: github.com/ortus-boxlang/bx-ai/examples
• Enterprise Services: ai.ortussolutions.com

Join the BoxLang community and help us advance AI development on the JVM!

About the Author

Luis Majano is CEO and Chief Software Architect at Ortus Solutions, where he leads the development of BoxLang and the broader enterprise tooling ecosystem. He created the ColdBox framework in 2006 and has spent nearly two decades building professional open source solutions for the JVM. Ortus Solutions operates as a Christian-based company with 40+ professionals and maintains 350+ open-source libraries.

About BoxLang

BoxLang is a modern, dynamic JVM programming language designed for enterprise application development. It provides 100% Java interoperability, CFML compatibility, and supports multi-runtime deployment across operating systems, web servers, serverless platforms, and WebAssembly. BoxLang is developed and maintained by Ortus Solutions without venture capital funding, following professional open-source principles.