The MemberJunction AI Core package provides a comprehensive abstraction layer for working with various AI models (LLMs, Video and Audio Generation, Text-To-Speech (TTS), embedding models, etc.) in a provider-agnostic way, allowing your application to easily switch between different AI providers without refactoring your code.
This package serves as the foundation for all AI capabilities in the MemberJunction ecosystem. It defines abstract base classes and interfaces that are implemented by provider-specific packages, enabling seamless integration with various AI services while maintaining a consistent API.
As part of a major type reorganization to improve code organization and reduce circular dependencies:
BaseModel, BaseLLM, BaseEmbeddings, etc.)BaseResult, ChatResult, ModelUsage, etc.)@memberjunction/ai-agents@memberjunction/ai-prompts@memberjunction/aiengineIMPORTANT: This package can be used completely independently from the rest of the MemberJunction framework:
The @memberjunction/ai package and all provider packages in @memberjunction/ai-* are designed to be lightweight, standalone modules that can be used in any TypeScript project.
npm install @memberjunction/ai
Then install one or more provider packages:
npm install @memberjunction/ai-openai
npm install @memberjunction/ai-anthropic
npm install @memberjunction/ai-mistral
# etc.
For maximum flexibility, use the class factory approach to select the provider at runtime:
import { BaseLLM, ChatParams } from '@memberjunction/ai';
import { MJGlobal } from '@memberjunction/global';
// Required to stop tree-shaking of the MistralLLM - since there's no static code path to this class when using Class Factory pattern, you need this to prevent some bundlers from tree-shaking optimization on this class.
import { LoadMistralLLM } from '@memberjunction/ai-mistral';
LoadMistralLLM();
// Get an implementation of BaseLLM by provider name
const llm = MJGlobal.Instance.ClassFactory.CreateInstance<BaseLLM>(
BaseLLM,
'MistralLLM', // Provider class name
'your-api-key'
);
// Use the abstracted interface
const params: ChatParams = {
model: 'mistral-large-latest',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is AI abstraction?' }
]
};
const result = await llm.ChatCompletion(params);
Use environment variables or configuration to select the provider:
import { BaseLLM } from '@memberjunction/ai';
import { MJGlobal } from '@memberjunction/global';
import dotenv from 'dotenv';
// Required to stop tree-shaking of the OpenAILLM - since there's no static code path to this class when using Class Factory pattern, you need this to prevent some bundlers from tree-shaking optimization on this class.
import { LoadOpenAILLM } from '@memberjunction/ai-openai';
LoadOpenAILLM();
dotenv.config();
const providerName = process.env.AI_PROVIDER || 'OpenAILLM';
const apiKey = process.env.AI_API_KEY;
const llm = MJGlobal.Instance.ClassFactory.CreateInstance<BaseLLM>(
BaseLLM,
providerName,
apiKey
);
When necessary, you can directly use a specific AI provider:
import { OpenAILLM } from '@memberjunction/ai-openai';
// Note with direct use of the OpenAILLM class no need for the LoadOpenAILLM call to prevent tree shaking since there is a static code path to the class
// Create an instance with your API key
const llm = new OpenAILLM('your-openai-api-key');
// Use the provider-specific implementation
const result = await llm.ChatCompletion({
model: 'gpt-4',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'What is AI abstraction?' }
]
});
console.log(result.data.choices[0].message.content);
The foundational abstract class for all AI models. Provides:
Abstract class for text generation models. Features:
Abstract class for text embedding models. Provides:
Abstract class for audio processing models. Supports:
Abstract class for video generation models. Enables:
Abstract class for image generation models (placeholder for future implementation)
For interactive conversations with AI models:
import { ChatParams, ChatResult, ChatMessage } from '@memberjunction/ai';
const params: ChatParams = {
model: 'your-model-name',
messages: [
{ role: 'system', content: 'System instruction' },
{ role: 'user', content: 'User message' },
{ role: 'assistant', content: 'Assistant response' }
],
temperature: 0.7,
maxOutputTokens: 1000
};
const result: ChatResult = await llm.ChatCompletion(params);
For real-time streaming of responses:
import { ChatParams, ChatResult, ChatMessage, StreamingChatCallbacks } from '@memberjunction/ai';
// Define the streaming callbacks
const callbacks: StreamingChatCallbacks = {
// Called when a new chunk arrives
OnContent: (chunk: string, isComplete: boolean) => {
if (isComplete) {
console.log("\nStream completed!");
} else {
// Print chunks as they arrive (or add to UI)
process.stdout.write(chunk);
}
},
// Called when the complete response is available
OnComplete: (finalResponse: ChatResult) => {
console.log("\nFull response:", finalResponse.data.choices[0].message.content);
console.log("Total tokens:", finalResponse.data.usage.totalTokens);
},
// Called if an error occurs during streaming
OnError: (error: any) => {
console.error("Streaming error:", error);
}
};
// Create streaming chat parameters
const params: ChatParams = {
model: 'gpt-4',
messages: [
{ role: 'system', content: 'You are a helpful assistant.' },
{ role: 'user', content: 'Write a short poem about AI, one line at a time.' }
],
streaming: true, // Enable streaming
streamingCallbacks: callbacks
};
// The ChatCompletion API remains the same, but will stream results
await llm.ChatCompletion(params);
The MemberJunction AI Core package supports cancellation of long-running operations using the standard JavaScript AbortSignal pattern. This provides a clean, standardized way to cancel AI operations when needed.
The AbortSignal pattern uses a separation of concerns approach:
AbortController and decides when to cancelAbortSignal token and handles how to cancelimport { ChatParams, BaseLLM } from '@memberjunction/ai';
async function cancellableAIChat() {
// Create the cancellation controller (the "boss")
const controller = new AbortController();
// Set up automatic timeout cancellation
const timeout = setTimeout(() => {
controller.abort(); // Cancel after 30 seconds
}, 30000);
try {
const params: ChatParams = {
model: 'gpt-4',
messages: [
{ role: 'user', content: 'Write a very long story...' }
],
cancellationToken: controller.signal // Pass the signal token
};
const result = await llm.ChatCompletion(params);
clearTimeout(timeout); // Clear timeout if completed successfully
return result;
} catch (error) {
if (error.message.includes('cancelled')) {
console.log('Operation was cancelled');
} else {
console.error('Operation failed:', error);
}
}
}
Perfect for UI applications where users can cancel operations:
class AIInterface {
private currentController: AbortController | null = null;
async startAIConversation() {
// Create new controller for this conversation
this.currentController = new AbortController();
try {
const result = await llm.ChatCompletion({
model: 'gpt-4',
messages: [{ role: 'user', content: 'Generate a detailed report...' }],
cancellationToken: this.currentController.signal
});
console.log('AI Response:', result.data.choices[0].message.content);
} catch (error) {
if (error.message.includes('cancelled')) {
console.log('User cancelled the conversation');
}
} finally {
this.currentController = null;
}
}
// Called when user clicks "Cancel" button
cancelConversation() {
if (this.currentController) {
this.currentController.abort(); // Instant cancellation
}
}
}
One signal can be cancelled from multiple sources:
async function smartAIExecution() {
const controller = new AbortController();
const signal = controller.signal;
// 1. User cancel button
document.getElementById('cancel')?.addEventListener('click', () => {
controller.abort(); // User cancellation
});
// 2. Resource limit cancellation
if (await checkMemoryUsage() > MAX_MEMORY) {
controller.abort(); // Resource limit cancellation
}
// 3. Window unload cancellation
window.addEventListener('beforeunload', () => {
controller.abort(); // Page closing cancellation
});
// 4. Timeout cancellation
setTimeout(() => controller.abort(), 60000); // 1 minute timeout
try {
const result = await llm.ChatCompletion({
model: 'gpt-4',
messages: [{ role: 'user', content: 'Complex analysis task...' }],
cancellationToken: signal // One token, many cancel sources!
});
return result;
} catch (error) {
// The AI operation doesn't know WHY it was cancelled - just that it should stop
console.log('AI operation was cancelled:', error.message);
}
}
The AI Core package implements cancellation at multiple levels:
// Internal implementation example (simplified)
async ChatCompletion(params: ChatParams): Promise<ChatResult> {
// Check if already cancelled before starting
if (params.cancellationToken?.aborted) {
throw new Error('Operation was cancelled');
}
// For providers with native cancellation support
if (this.hasNativeCancellation) {
return await this.callProviderAPI({
...params,
signal: params.cancellationToken // Native fetch cancellation
});
}
// Fallback: Promise.race pattern for other providers
const promises = [
this.callProviderAPI(params) // The actual AI call
];
if (params.cancellationToken) {
promises.push(
new Promise<never>((_, reject) => {
params.cancellationToken!.addEventListener('abort', () => {
reject(new Error('Operation was cancelled'));
});
})
);
}
return await Promise.race(promises);
}
The cancellation token pattern provides a robust, standardized way to make AI operations responsive and resource-efficient!
For summarizing longer text content:
import { SummarizeParams, SummarizeResult } from '@memberjunction/ai';
const params: SummarizeParams = {
text: 'Long text to summarize...',
model: 'your-model-name',
maxWords: 100
};
const result: SummarizeResult = await llm.SummarizeText(params);
console.log(result.summary);
For categorizing text into predefined classes:
import { ClassifyParams, ClassifyResult } from '@memberjunction/ai';
const params: ClassifyParams = {
text: 'Text to classify',
model: 'your-model-name',
classes: ['Category1', 'Category2', 'Category3']
};
const result: ClassifyResult = await llm.ClassifyText(params);
console.log(result.classification);
Control the format of AI responses:
const params: ChatParams = {
// ...other parameters
responseFormat: 'JSON', // 'Any', 'Text', 'Markdown', 'JSON', or 'ModelSpecific'
};
// For provider-specific response formats
const customFormatParams: ChatParams = {
// ...other parameters
responseFormat: 'ModelSpecific',
modelSpecificResponseFormat: {
// Provider-specific format options
}
};
The MemberJunction AI Core package now provides rich, structured error information to enable intelligent retry logic and provider failover. All operations return results extending BaseResult which now includes detailed error information.
const result = await llm.ChatCompletion(params);
if (!result.success) {
console.error('Error:', result.errorMessage);
console.error('Status Text:', result.statusText);
console.error('Exception:', result.exception);
console.error('Time Elapsed:', result.timeElapsed, 'ms');
} else {
console.log('Success! Response time:', result.timeElapsed, 'ms');
}
const result = await llm.ChatCompletion(params);
if (!result.success && result.errorInfo) {
const { errorInfo } = result;
console.log(`Error Type: ${errorInfo.errorType}`);
console.log(`HTTP Status: ${errorInfo.httpStatusCode}`);
console.log(`Severity: ${errorInfo.severity}`);
console.log(`Can Failover: ${errorInfo.canFailover}`);
// Handle based on error type
switch (errorInfo.errorType) {
case 'RateLimit':
// Wait and retry or switch providers
const delay = errorInfo.suggestedRetryDelaySeconds || 30;
console.log(`Rate limited. Retry after ${delay} seconds`);
break;
case 'Authentication':
// Fatal error - check API keys
console.error('Authentication failed. Check API key configuration.');
break;
case 'ServiceUnavailable':
// Try another provider
if (errorInfo.canFailover) {
console.log('Service unavailable. Switching to backup provider...');
}
break;
}
}
The package categorizes errors into the following types:
RateLimit: Rate limit exceeded (HTTP 429). Suggests switching providers or waitingAuthentication: Auth failure (HTTP 401/403). Usually indicates invalid API keyServiceUnavailable: Service down (HTTP 503). Provider temporarily unavailableInternalServerError: Server error (HTTP 500). Problem on provider's sideNetworkError: Connection issues, timeouts, DNS failuresInvalidRequest: Bad request format (HTTP 400). Problem with request parametersModelError: Model-specific issues (not found, overloaded)Unknown: Unclassified errorsErrors are classified by severity to guide retry strategies:
Transient: Temporary error that may resolve with immediate retryRetriable: Error requiring waiting or provider switching before retryFatal: Permanent error that won't be resolved by retryingThe package includes an ErrorAnalyzer utility for providers to use:
import { ErrorAnalyzer } from '@memberjunction/ai';
try {
// Provider API call
} catch (error) {
const errorInfo = ErrorAnalyzer.analyzeError(error, 'OpenAI');
// errorInfo now contains structured error details
}
import { BaseLLM, ChatParams, AIErrorInfo } from '@memberjunction/ai';
class ResilientAIClient {
private providers: Array<{ name: string; llm: BaseLLM }> = [];
async chatWithFailover(params: ChatParams): Promise<ChatResult> {
for (const provider of this.providers) {
try {
const result = await provider.llm.ChatCompletion(params);
if (result.success) {
return result;
}
// Check if we should try another provider
if (result.errorInfo?.canFailover) {
console.log(`Provider ${provider.name} failed. Trying next...`);
continue;
} else {
// Fatal error - don't try other providers
return result;
}
} catch (error) {
console.error(`Provider ${provider.name} threw exception:`, error);
}
}
throw new Error('All providers failed');
}
}
async function retryWithBackoff(
operation: () => Promise<ChatResult>,
maxRetries: number = 3
): Promise<ChatResult> {
let lastResult: ChatResult;
for (let attempt = 1; attempt <= maxRetries; attempt++) {
lastResult = await operation();
if (lastResult.success) {
return lastResult;
}
const errorInfo = lastResult.errorInfo;
if (!errorInfo || errorInfo.severity === 'Fatal') {
// Don't retry fatal errors
return lastResult;
}
// Calculate delay
const delay = errorInfo.suggestedRetryDelaySeconds ||
Math.pow(2, attempt - 1) * 1000; // Exponential backoff
console.log(`Retry ${attempt}/${maxRetries} after ${delay}s...`);
await new Promise(resolve => setTimeout(resolve, delay * 1000));
}
return lastResult!;
}
Monitor token usage consistently across different providers:
const result = await llm.ChatCompletion(params);
console.log('Prompt Tokens:', result.data.usage.promptTokens);
console.log('Completion Tokens:', result.data.usage.completionTokens);
console.log('Total Tokens:', result.data.usage.totalTokens);
The following provider packages implement the MemberJunction AI abstractions:
@memberjunction/ai-openai - OpenAI (GPT models)@memberjunction/ai-anthropic - Anthropic (Claude models)@memberjunction/ai-mistral - Mistral AI@memberjunction/ai-gemini - Google's Gemini models@memberjunction/ai-vertex - Google Vertex AI (various models including Gemini, others)@memberjunction/ai-bedrock - Amazon Bedrock (Claude, Llama, Titan, etc.)@memberjunction/ai-groq - Groq's optimized inference (https://www.groq.com)@memberjunction/ai-bettybot - Betty AI (https://www.meetbetty.ai)@memberjunction/ai-azure - Azure AI Foundry with support for OpenAI, Mistral, Phi, more@memberjunction/ai-cerebras - Cerebras models@memberjunction/ai-elevenlabs - ElevenLabs audio models@memberjunction/ai-heygen - HeyGen video modelsNote: Each provider implements the features they support. See individual provider documentation for specific capabilities.
The BaseLLM class uses a template method pattern for handling streaming:
ChatCompletion method checks if streaming is requested and supportedhandleStreamingChatCompletioncreateStreamingRequest: Creates the provider-specific streaming requestprocessStreamingChunk: Processes individual chunks from the streamfinalizeStreamingResponse: Creates the final response objectThis architecture allows for a clean separation between common streaming logic and provider-specific implementations.
// Import base model classes
import { BaseLLM, BaseEmbeddings, BaseAudioGenerator } from '@memberjunction/ai';
// Import result types
import { ChatResult, ModelUsage, BaseResult } from '@memberjunction/ai';
// Import parameter types
import { ChatParams, ChatMessage, StreamingChatCallbacks } from '@memberjunction/ai';
// Import utility classes
import { AIAPIKeys, GetAIAPIKey } from '@memberjunction/ai';
@memberjunction/global - MemberJunction global utilities including class factoryrxjs - Reactive extensions for JavaScriptThe Core package exports fundamental types used throughout the AI ecosystem:
BaseModel - Foundation class for all AI modelsBaseLLM - Base class for language modelsBaseEmbeddings - Base class for embedding modelsBaseAudioGenerator - Base class for audio modelsBaseVideoGenerator - Base class for video modelsBaseDiffusion - Base class for image generation modelsBaseResult - Base result structure for all AI operationsChatResult - Result from chat completionsModelUsage - Token/resource usage trackingSummarizeResult - Text summarization resultsClassifyResult - Text classification resultsEmbeddingResult - Embedding generation resultsChatMessage - Message structure for conversationsChatParams - Parameters for chat operationsStreamingChatCallbacks - Callbacks for streaming responsesParallelChatCompletionsCallbacks - Callbacks for parallel executionAll operations return results extending BaseResult:
class BaseResult {
success: boolean;
startTime: Date;
endTime: Date;
errorMessage: string;
exception: any;
errorInfo?: AIErrorInfo; // Enhanced error details (v2.47.0+)
timeElapsed: number; // Computed getter
}
// Enhanced error information structure
interface AIErrorInfo {
httpStatusCode?: number; // HTTP status code (429, 500, etc.)
errorType: AIErrorType; // Categorized error type
severity: ErrorSeverity; // Transient, Retriable, or Fatal
suggestedRetryDelaySeconds?: number; // Provider-suggested retry delay
canFailover: boolean; // Whether switching providers might help
providerErrorCode?: string; // Original provider error code
context?: Record<string, any>; // Additional error context
}
Extends BaseResult with chat-specific data:
class ChatResult extends BaseResult {
data: {
choices: Array<{
message: ChatCompletionMessage;
index: number;
finishReason?: string;
}>;
usage: ModelUsage;
};
statusText?: string;
}
Supports multi-modal content and optional typed metadata:
type ChatMessage<M = any> = {
role: 'system' | 'user' | 'assistant';
content: string | ChatMessageContentBlock[];
metadata?: M; // Optional typed metadata for extended functionality
}
type ChatMessageContentBlock = {
type: 'text' | 'image_url' | 'video_url' | 'audio_url' | 'file_url';
content: string; // URL or base64 encoded content
}
Generic Metadata Parameter:
The ChatMessage type is now generic, allowing you to attach typed metadata to messages:
any for backward compatibilityChatMessage<MyMetadata> to specify a custom metadata typeChatMessage<AgentChatMessageMetadata> to track message expiration, compaction, and expansion stateinterface StreamingChatCallbacks {
OnContent?: (chunk: string, isComplete: boolean) => void;
OnComplete?: (finalResponse: ChatResult) => void;
OnError?: (error: any) => void;
}
interface ParallelChatCompletionsCallbacks {
OnCompletion?: (response: ChatResult, index: number) => void;
OnError?: (error: any, index: number) => void;
OnAllCompleted?: (responses: ChatResult[]) => void;
}
The package includes a flexible API key management system through the AIAPIKeys class:
import { GetAIAPIKey } from '@memberjunction/ai';
// Get API key for a specific provider
const apiKey = GetAIAPIKey('OpenAI');
By default, it looks for environment variables with the pattern: AI_VENDOR_API_KEY__[PROVIDER_NAME]
Example:
AI_VENDOR_API_KEY__OPENAI=your-api-key
AI_VENDOR_API_KEY__ANTHROPIC=your-api-key
AI_VENDOR_API_KEY__MISTRAL=your-api-key
Advanced Authentication: When using the full MemberJunction framework with
@memberjunction/ai-prompts, you can leverage the Credentials system for encrypted, audited credential management with hierarchical resolution. See the AI Authentication Guide for details on:
- Credential resolution hierarchy (vendor → model → prompt → request)
- Integration with the encrypted Credentials system
- Support for complex authentication schemes (Azure, AWS, OAuth)
- Migration path from environment variables
You can extend the AIAPIKeys class to implement custom API key retrieval logic:
import { AIAPIKeys, RegisterClass } from '@memberjunction/ai';
@RegisterClass(AIAPIKeys, 'CustomAPIKeys', 2) // Priority 2 overrides default
export class CustomAPIKeys extends AIAPIKeys {
public GetAPIKey(AIDriverName: string): string {
// Your custom logic here
// Could retrieve from database, vault, etc.
return super.GetAPIKey(AIDriverName); // Fallback to default
}
}
As of v2.50.0, you can provide API keys at runtime without modifying environment variables or global configuration. This is useful for multi-tenant applications or testing with different API keys.
When creating AI model instances directly, pass the API key to the constructor:
import { OpenAILLM } from '@memberjunction/ai-openai';
const llm = new OpenAILLM('sk-your-runtime-api-key');
When using the class factory pattern, the API key is passed as the second parameter:
import { BaseLLM } from '@memberjunction/ai';
import { MJGlobal } from '@memberjunction/global';
const llm = MJGlobal.Instance.ClassFactory.CreateInstance<BaseLLM>(
BaseLLM,
'OpenAILLM',
'sk-your-runtime-api-key' // Runtime API key
);
The package exports an AIAPIKey interface for structured API key configuration:
export interface AIAPIKey {
/**
* The driver class name (e.g., 'OpenAILLM', 'AnthropicLLM', 'GroqLLM')
* This should match the exact driver class name used by the AI provider
*/
driverClass: string;
/**
* The API key value for the specified driver class
*/
apiKey: string;
}
Many providers support additional configuration beyond the API key:
const llm = new SomeLLM('api-key');
llm.SetAdditionalSettings({
baseURL: 'https://custom-endpoint.com',
organization: 'my-org',
// Provider-specific settings
});
While this package can be used standalone, it integrates seamlessly with the MemberJunction framework:
@memberjunction/global for class factory pattern and registration@memberjunction/global (^2.43.0) - MemberJunction global utilities including class factoryrxjs (^7.8.1) - Reactive extensions for JavaScript (used in streaming implementations)dotenv (^16.4.1) - Environment variable managementtypeorm (^0.3.20) - ORM functionality (optional, only if using with full MemberJunction)cd packages/AI/Core
npm run build
The package is configured with TypeScript strict mode and targets ES2022. See tsconfig.json for full compiler options.
result.success before accessing data"API key cannot be empty" error
Provider not found when using class factory
Streaming not working
llm.SupportsStreaming)Type errors with content blocks
The ChatParams class supports the following parameters for controlling LLM behavior:
model (required): The model name to usetemperature: Controls randomness (0.0 = deterministic, 2.0 = very random)maxOutputTokens: Maximum tokens to generate in responseresponseFormat: Output format - 'Any', 'Text', 'Markdown', 'JSON', or 'ModelSpecific'seed: Random seed for reproducible outputs (provider-dependent)stopSequences: Array of sequences that will stop generationtopP: Top-p (nucleus) sampling (0-1). Alternative to temperature, considers cumulative probabilitytopK: Top-k sampling. Limits to top K most likely tokens (provider-dependent)minP: Minimum probability threshold (0-1). Filters out low-probability tokensfrequencyPenalty: Reduce token repetition based on frequency (-2.0 to 2.0)presencePenalty: Encourage topic diversity (-2.0 to 2.0)streaming: Enable streaming responsesincludeLogProbs: Request log probabilities for tokenstopLogProbs: Number of top log probabilities to return (2-20)effortLevel: Model-specific effort/reasoning levelreasoningBudgetTokens: Token budget for reasoning modelsenableCaching: Enable provider caching featurescancellationToken: AbortSignal for cancelling operationsNot all providers support all parameters. The framework will:
Common support patterns:
See the repository root for license information.