ptesny/.Mistral AI and SAP Kyma serverless (nodejs) story.md Secret

## .Mistral AI and SAP Kyma serverless (nodejs) story.md

      
    Raw
  

              .Mistral AI and SAP Kyma serverless (nodejs) story.md
            
          
    Mistral gagnant. Mistral AI and SAP Kyma serverless (nodejs) story


Open and portable generative AI.

SAP BTP, Kyma runtime

This is a companion gist to Mistral gagnant. Mistral AI and SAP Kyma serverless. | SAP Blogs.

Let me tell you my story.
I have come across Mistral AI as an alternative to OpenAI/SAP AI core.

And, very naturally, I wanted to be able to consume the Mistral Javascript Client from a backend service running on SAP BTP, Kyma runtime (which is the SAP's managed and business-friendly kubernetes offering) as a serverless function.
And here is how the story unfolded...


Table of Contents

	 Tackling SAP Kyma serverless nodejs functions CommonJS (cjs) vs. ECMAScript (esm) conundrum
 	The making of.

 	SyntaxError: Cannot use import statement outside a module .
 	ERR_REQUIRE_ESM error when trying to require('@mistralai/mistralai').
 	Dynamic import..


 	Mistral gagnant.

 	mistral.mjs.
 	handler.js.
 	tadam.


	Integration into SAP APIM
	Integration into SAP Build Apps with the BTP destinations


1. Tackling SAP Kyma serverless nodejs functions CommonJS (cjs) vs. ECMAScript (esm) conundrum.

Albeit "tackling" is very much a soccer term, it applies quite well to the dealing with the CJS verus ESM divide.

That, especially, in the context of SAP Kyma serverless.
A kyma nodejs function (either inline or git repository based) requires at minima a handler.js file which adheres to the CommonJS module standard.

(CommonJS modules are the genuine way of packaging the JavaScript code for Node.js in applications implementing backend services.)
On the other hand, the MistratAI nodejs client is implemented as an ES-standard module so it can be run in a browser.

(In layman terms, it simply means one cannot either require or import a MistralAI nodejs client package into a CJS module.)
In lieu, it must be imported into an ESM (ECMAScript) module.
2. The making of.

2.1. SyntaxError: Cannot use import statement outside a module.

import MistralClient from '@mistralai/mistralai'; cannot work from a CJS module.
> nodejs18-runtime@0.1.0 start
> node server.js
user code load error: SyntaxError: Cannot use import statement outside a module
Content loaded is not a function /usr/src/app/function/mistralai.js:1
import MistralClient from '@mistralai/mistralai';
^^^^^^
SyntaxError: Cannot use import statement outside a module
at internalCompileFunction (node:internal/vm:73:18)
at wrapSafe (node:internal/modules/cjs/loader:1178:20)
at Module._compile (node:internal/modules/cjs/loader:1220:27)
at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
at Module.load (node:internal/modules/cjs/loader:1119:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at Module.require (node:internal/modules/cjs/loader:1143:19)
at Hook._require.Module.require (/usr/src/app/node_modules/require-in-the-middle/index.js:101:39)
at Hook._require.Module.require (/usr/src/app/node_modules/@opentelemetry/instrumentation-express/node_modules/require-in-the-middle/index.js:188:39)
at require (node:internal/modules/cjs/helpers:110:18)

2.2. ERR_REQUIRE_ESM error when trying to require('@mistralai/mistralai').

const MistralClient = require('@mistralai/mistralai') the require does not work either
> nodejs18-runtime@0.1.0 start
> node server.js
user code load error: Error [ERR_REQUIRE_ESM]: require() of ES Module /usr/src/app/function/node_modules/@mistralai/mistralai/src/client.js from /usr/src/app/function/handler.js not supported.
Instead change the require of client.js in /usr/src/app/function/handler.js to a dynamic import() which is available in all CommonJS modules.
Content loaded is not a function Error [ERR_REQUIRE_ESM]: require() of ES Module /usr/src/app/function/node_modules/@mistralai/mistralai/src/client.js from /usr/src/app/function/handler.js not supported.
Instead change the require of client.js in /usr/src/app/function/handler.js to a dynamic import() which is available in all CommonJS modules.
at Hook._require.Module.require (/usr/src/app/node_modules/require-in-the-middle/index.js:101:39)
at Hook._require.Module.require (/usr/src/app/node_modules/@opentelemetry/instrumentation-express/node_modules/require-in-the-middle/index.js:188:39)
at Object.<anonymous> (/usr/src/app/function/handler.js:6:23)
at Hook._require.Module.require (/usr/src/app/node_modules/require-in-the-middle/index.js:101:39)
at Hook._require.Module.require (/usr/src/app/node_modules/@opentelemetry/instrumentation-express/node_modules/require-in-the-middle/index.js:188:39)
at loadFunction (/usr/src/app/server.js:45:15)
at Object.<anonymous> (/usr/src/app/server.js:176:12) {
code: 'ERR_REQUIRE_ESM'
}

2.3. Dynamic import.

Despite, the encouraging comment that a dynamic import could be used as a mechanims to address this CJS/EMS require/import conundrum, it simply did not work.
const MistralClient = await import('@mistralai/mistralai');
> nodejs18-runtime@0.1.0 start
> node server.js
user code load error: SyntaxError: await is only valid in async functions and the top level bodies of modules
Content loaded is not a function /usr/src/app/function/handler.js:10
const MistralClient = await import('@mistralai/mistralai');
^^^^^
SyntaxError: await is only valid in async functions and the top level bodies of modules
at internalCompileFunction (node:internal/vm:73:18)
at wrapSafe (node:internal/modules/cjs/loader:1178:20)
at Module._compile (node:internal/modules/cjs/loader:1220:27)
at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
at Module.load (node:internal/modules/cjs/loader:1119:32)
at Module._load (node:internal/modules/cjs/loader:960:12)
at Module.require (node:internal/modules/cjs/loader:1143:19)
at Hook._require.Module.require (/usr/src/app/node_modules/require-in-the-middle/index.js:101:39)
at Hook._require.Module.require (/usr/src/app/node_modules/@opentelemetry/instrumentation-express/node_modules/require-in-the-middle/index.js:188:39)
at require (node:internal/modules/cjs/helpers:110:18)

Then, I have tried to moved the const MistralClient = await import('@mistralai/mistralai'); into an async function body but that did not help either.
Last but not least, I have come up across this fantastic blogpost by Adam Coster.
3. Mistral gagnant.

Eventually, I was able to make it work.  I created another source file mistralai.mjs with the following import code:
3.1 mistral.mjs

// https://www.npmjs.com/package/@mistralai/mistralai
//
import MistralClient from '@mistralai/mistralai';

const MISTRAL_API_KEY = '<MISTRAL_API_KEY>'; 
const apiKey = process.env.MISTRAL_API_KEY || MISTRAL_API_KEY;
export const client = new MistralClient(apiKey);
export const msg = "Hello world!"; // this is for a test

3.2 handler.js

async function mistralai_api(event) {
// https://www.npmjs.com/package/@mistralai/mistralai
// https://adamcoster.com/blog/commonjs-and-esm-importexport-compatibility-examples

   const MistralClientModule = await import('./mistralai.mjs');
   console.log('MistralClientModule: ', MistralClientModule);
   console.log('MistralClientModule.msg: ', MistralClientModule.msg);
   const client = MistralClientModule.client;
}

3.3 tadam

client: MistralClient {
  _request: [AsyncFunction: _request],
  _makeChatCompletionRequest: [Function: _makeChatCompletionRequest],
  listModels: [AsyncFunction: listModels],
  chat: [AsyncFunction: chat],
  chatStream: [AsyncGeneratorFunction: chatStream],
  embeddings: [AsyncFunction: embeddings],
  endpoint: 'https://api.mistral.ai',
  apiKey: '********',
  maxRetries: 5,
  timeout: 120,
  modelDefault: 'mistral'
}

4. Integration into SAP APIM.


Request body.


{
  "model": "mistral-small-latest",
  "messages": [
    {
      "role": "user",
      "content": "Who is the best French dish? Answer in one short sentence."
    }
  ],
  "temperature": 0.7,
  "top_p": 1,
  "max_tokens": 512,
  "stream": true,
  "safe_prompt": false,
  "random_seed": 1337
}

response body


"Boeuf Bourguignon is a beloved French dish, featuring tender beef slow-cooked in red wine with vegetables and bacon."
5. Integration into SAP Build Apps with the BTP destinations.

In order to access the backend service on SAP Kyma the simplest is to use a BTP destination.

Additional resources


CommonJS (cjs) and Modules (esm): Import compatibility
JS Client library for Mistral AI platform | github
Mistral Javascript Client | npm


## handler.js
async function mistralai_api(event) {
// https://www.npmjs.com/package/@mistralai/mistralai
// https://adamcoster.com/blog/commonjs-and-esm-importexport-compatibility-examples

   const MistralClientModule = await import('./mistralai.mjs');
   console.log('MistralClientModule: ', MistralClientModule);
   console.log('MistralClientModule.msg: ', MistralClientModule.msg);
   const client = MistralClientModule.client;
}

module.exports = {
  main: async function (event, context) {

    const req = event.extensions.request;

    const message = `Hello World`
      + ` from the Kyma Function ${context['function-name']}`
      + ` running on ${context.runtime}!`
      + ` with the request headers ${JSON.stringify(req.headers,0,2)}`;
    console.log(message);

    if (typeof req.path !== undefined) {
      console.log('path: ', JSON.stringify(req.path,0,2))
    }
    if (typeof req.params !== undefined) {
      console.log('params: ', JSON.stringify(req.params,0,2))
      console.log('params.filepath: ', JSON.stringify(req.params.filepath,0,2))

    }
    if (typeof req.url !== undefined) {
      console.log('url: ', JSON.stringify(req.url,0,2))
    }
    if (typeof req.authInfo !== undefined) {
      console.log('authInfo: ', JSON.stringify(req.authInfo,0,2))
    }
    if (typeof req.body !== undefined) {
      console.log('body: ', JSON.stringify(req.body,0,2))
    }

    const { pathname } = new URL(req.url || '', `https://${req.headers.host}`)
    console.log('pathname: ', pathname)

    const url = require("url");
    var url_parts = url.parse(req.url);
    console.log(url_parts);
    console.log(url_parts.pathname);

    // returns an array with paths
    const path_array = req.url.match('^[^?]*')[0].split('/').slice(1);
    console.log(path_array)

    console.log(req.url.match('^[^?]*')[0])


    if (path_array[0] == 'mistralai') {
      return mistralai_api(event);
    }
  }
}

## mistralai.mjs
// https://www.npmjs.com/package/@mistralai/mistralai
//
import MistralClient from '@mistralai/mistralai';

const MISTRAL_API_KEY = '<MISTRAL_API_KEY>';
const apiKey = process.env.MISTRAL_API_KEY || MISTRAL_API_KEY;
export const client = new MistralClient(apiKey);
export const msg = "Hello world!"; // this is for a test

## package.json
{
  "name": "mistral-gagnant",
  "version": "1.1.0",

   "dependencies": {
    "axios": "latest",
    "@kubernetes/client-node": "latest",
    "@mistralai/mistralai": "latest",
    "lodash": "latest",
    "cheerio": "latest"
  },
  "devDependencies": {
  }
}
	async function mistralai_api(event) {
	// https://www.npmjs.com/package/@mistralai/mistralai
	// https://adamcoster.com/blog/commonjs-and-esm-importexport-compatibility-examples

	const MistralClientModule = await import('./mistralai.mjs');
	console.log('MistralClientModule: ', MistralClientModule);
	console.log('MistralClientModule.msg: ', MistralClientModule.msg);
	const client = MistralClientModule.client;
	}

	module.exports = {
	main: async function (event, context) {

	const req = event.extensions.request;

	const message = `Hello World`
	+ ` from the Kyma Function ${context['function-name']}`
	+ ` running on ${context.runtime}!`
	+ ` with the request headers ${JSON.stringify(req.headers,0,2)}`;
	console.log(message);

	if (typeof req.path !== undefined) {
	console.log('path: ', JSON.stringify(req.path,0,2))
	}
	if (typeof req.params !== undefined) {
	console.log('params: ', JSON.stringify(req.params,0,2))
	console.log('params.filepath: ', JSON.stringify(req.params.filepath,0,2))

	}
	if (typeof req.url !== undefined) {
	console.log('url: ', JSON.stringify(req.url,0,2))
	}
	if (typeof req.authInfo !== undefined) {
	console.log('authInfo: ', JSON.stringify(req.authInfo,0,2))
	}
	if (typeof req.body !== undefined) {
	console.log('body: ', JSON.stringify(req.body,0,2))
	}

	const { pathname } = new URL(req.url \|\| '', `https://${req.headers.host}`)
	console.log('pathname: ', pathname)

	const url = require("url");
	var url_parts = url.parse(req.url);
	console.log(url_parts);
	console.log(url_parts.pathname);

	// returns an array with paths
	const path_array = req.url.match('^[^?]*')[0].split('/').slice(1);
	console.log(path_array)

	console.log(req.url.match('^[^?]*')[0])


	if (path_array[0] == 'mistralai') {
	return mistralai_api(event);
	}
	}
	}
	// https://www.npmjs.com/package/@mistralai/mistralai
	//
	import MistralClient from '@mistralai/mistralai';

	const MISTRAL_API_KEY = '<MISTRAL_API_KEY>';
	const apiKey = process.env.MISTRAL_API_KEY \|\| MISTRAL_API_KEY;
	export const client = new MistralClient(apiKey);
	export const msg = "Hello world!"; // this is for a test
	{
	"name": "mistral-gagnant",
	"version": "1.1.0",

	"dependencies": {
	"axios": "latest",
	"@kubernetes/client-node": "latest",
	"@mistralai/mistralai": "latest",
	"lodash": "latest",
	"cheerio": "latest"
	},
	"devDependencies": {
	}
	}