Skip to content

Instantly share code, notes, and snippets.

@pkdone

pkdone/mongodb-csfle-example-kmip-or-local-key.md

Last active July 6, 2023 03:52
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save pkdone/084c91e0f99b46aae23919182284bfd5 to your computer and use it in GitHub Desktop.
Save pkdone/084c91e0f99b46aae23919182284bfd5 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
mongodb-csfle-example-kmip-or-local-key.md

MongoDB Client-Side Field Level Encryption (CSFLE) Using KMIP or Local Master Key (with mongosh)

Assumptions

  • You have a MongoDB Enterprise deployment already running and accessible (self-managed or in Atlas)
  • You have the MongoDB Enterprise binary mongocryptd accessibe on your system path to enable automated encryption
  • You have the modern MongoDB Shell (mongosh) installed locally on your workstation
  • You have a KMIP Server running and accessible, if you don't intend to use a local keyfile (for an example of running and configuring a Hashicorp Vault development instance, see: Hashicorp Vault Configuration For MongoDB KMIP Use)

Configure Local Workstation Context Files

From a terminal, execute the code below after first:

  • Changing useLocalMasterKey to true if you don't want to use a KMIP managed master key and instead want to use a local keyfile
  • Changing url to match the cluster address of your MongoDB deployment
  • Changing kmipServer to match the address of your KMIP deployment (only necessary if not using a local keyfile)
# Clean out previously created files, if any
rm -f master_local_keyfile data*_key_id_file init_env.js people_schema.js

# Generate local master keyfile for CSFLE (not needed when using KMIP, but doesn't do any harm to execute anyway)
echo $(head -c 96 /dev/urandom | base64 | tr -d '\n') > master_local_keyfile

# Initialise empty files which will later hold ids of data keys used for field encryption
touch data1_key_id_file data2_key_id_file

# Create a JS file to be used to load constant values 
tee init_env.js <<EOF
var useLocalMasterKey = false;
var url = "mongodb+srv://myuser:mypassd@mycluster.a12z.mongodb.net/";
var kmipServer = "localhost:5696"
var localMasterkeyFile = "master_local_keyfile";
var dataKey1IdFile = "data1_key_id_file";
var dataKey2IdFile = "data2_key_id_file";
var dataKey1Name = "ssnEncKey"
var dataKey2Name = "contactEncKey"
var dataKeysDBName = "csfle";
var dataKeysCollName = "dataKeys";
var dataDBName = "personsdb";
var dataCollName = "people";
var localMasterKey = fs.readFileSync(localMasterkeyFile).toString().trim();
var dataKey1IdText = fs.readFileSync(dataKey1IdFile).toString().trim();
var dataKey2IdText = fs.readFileSync(dataKey2IdFile).toString().trim();
EOF

Define a JSON Schema to Represent a Person with Some Fields Marked as Encrypted

Execute from a terminal:

# Create a the JSON schema file for a person
tee people_schema.js <<EOF
// Defines a JSON schema variable - IMPORTANT: variables dataKey1Id & dataKey1Id must already exist before loading this file
var peopleSchema = {
   "bsonType": "object",
   "properties": {
      "firstName": {"bsonType": "string"},
      "lastName":  {"bsonType": "string"},
      "ssn": {
         "encrypt": {
            "bsonType": "string",
            "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Random",
            "keyId": [dataKey1Id]
         }
      },
      "address": {
         "bsonType": "object",
         "properties": {
            "street": {bsonType: "string"},
            "city": {bsonType: "string"},
            "state": {bsonType: "string"},
            "zip": {bsonType: "string"},
         }
      },
      "contact": {
         "bsonType": "object",
         "properties": {
            "email"   : {
               "encrypt": {
                  "bsonType": "string",
                  "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
                  "keyId": [dataKey2Id]
               }
            },
            "mobile"  : {
               "encrypt": {
                  "bsonType": "string",
                  "algorithm": "AEAD_AES_256_CBC_HMAC_SHA_512-Deterministic",
                  "keyId": [dataKey2Id]
               }
            }
         },
      },
   }
};
EOF

NOTE: The SSN field doesn't actually need to use randomised encryption because the cardinality is naturally high, meaning there's no conceivable common use case where a frequency attack can be used against it. It is used here just to show an alternative example, but in the real world, it would be desirable to use a deterministic encryption approach to improve queryability.

Generate CSFLE Data Keys in a MongoDB Vault Using External KMIP or Local Master Key

From a terminal, execute the code below after first:

  • If using KMIP, changing "ca.pem", if required, to match the path of the Certificate Authority certificate file used by your KMIP server
  • If using KMIP, changing "client.pem", if required, to match the path of the Private Key + Certificate file to be used to access the KMIP server
mongosh --nodb

// Load constants from file
load("init_env.js");

// Assemble metadata for connecting to MongoDB using CSFLE
var clientSideFieldLevelEncryptionOptions = {};

// Use either local master key file or KMIP managed master key
if (useLocalMasterKey) {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "local": {
        "key": BinData(0, localMasterKey)
      }
    }
  };
} else {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "kmip": {
        "endpoint": kmipServer      
      },    
    },
    tlsOptions: {
      kmip: {
        tlsCAFile: "ca.pem",
        tlsCertificateKeyFile: "client.pem",
      }
    }
  };
}

// Connect to MongoDB with CSFLE params
var connection = Mongo(url, clientSideFieldLevelEncryptionOptions);

// Create a data key vault and persist 2 new generated data keys for encrypting fields differently
var keyVault = connection.getKeyVault();

if (useLocalMasterKey) {
    keyVault.createKey("local", {}, [dataKey1Name]);
    keyVault.createKey("local", {}, [dataKey2Name]);
} else {
    keyVault.createKey("kmip", {}, [dataKey1Name]);
    keyVault.createKey("kmip", {}, [dataKey2Name]);
}

// Get the IDs used to identify the 2 new data keys and save them to the local filesystem for future reference
var dataKey1Id = keyVault.getKeyByAltName(dataKey1Name).toArray()[0]._id;
var dataKey2Id = keyVault.getKeyByAltName(dataKey2Name).toArray()[0]._id;
fs.writeFileSync(dataKey1IdFile, dataKey1Id.toUUID().toString());
fs.writeFileSync(dataKey2IdFile, dataKey2Id.toUUID().toString());

// Load the schema for the people collection and associate it with the collection to enforce 
// server-side validation which marks some fields to be encrypted by the data keys
load("people_schema.js");
var db = connection.getDB(dataDBName);
db.createCollection(dataCollName, {"validator": {"$jsonSchema": peopleSchema}});

exit();

Test Inserting Records with Encrypted Fields

From a terminal, execute the code below after first:

  • If using KMIP, changing "ca.pem", if required, to match the path of the Certificate Authority certificate file used by your KMIP server
  • If using KMIP, changing "client.pem", if required, to match the path of the Private Key + Certificate file to be used to access the KMIP server
mongosh --nodb

// Load constants from file, data key ids from files and people schema from file
load("init_env.js");
var dataKey1Id = UUID(dataKey1IdText);
var dataKey2Id = UUID(dataKey2IdText);
load("people_schema.js");

// Get the people schema into an object that associates it with a specific database collection
var schemaMap = {};
schemaMap[dataDBName + "." + dataCollName] = peopleSchema;

// Assemble metadata for connecting to MongoDB using CSFLE, including client-side schema definition
var clientSideFieldLevelEncryptionOptions = {};

// Use either local master key file or KMIP managed master key
if (useLocalMasterKey) {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "local": {
        "key": BinData(0, localMasterKey)
      }
    },
    "schemaMap": schemaMap        
  };
} else {
  clientSideFieldLevelEncryptionOptions = {
    "keyVaultNamespace": dataKeysDBName + "." + dataKeysCollName,
    "kmsProviders": {
      "kmip": {
        "endpoint": kmipServer      
      },    
    },
    tlsOptions: {
      kmip: {
        tlsCAFile: "ca.pem",
        tlsCertificateKeyFile: "client.pem",
      }
    },
    "schemaMap": schemaMap        
  };
}

// Connect to MongoDB with CSFLE params and get handle onto main data DB
var connection = Mongo(url, clientSideFieldLevelEncryptionOptions);
var db = connection.getDB(dataDBName);

// Insert a person record
db[dataCollName].insertOne({
   firstName: 'Alan',
   lastName:  'Turing',
   ssn:       '901-01-0001',
   address: {
      street: '123 Main',
      city: 'Omaha',
      zip: '90210'
   },
   contact: {
      mobile: '202-555-1212',
      email:  'alan@example.com'
   }
});

// Show current persons records
db[dataCollName].find();

exit();

In the output above you should see the real unencrypted values of the fields ssn, mobile and email.

Use a Different MongoDB Shell with no CSFLE Awareness to View the Persisted Data

From a terminal, execute the code below after first changing the shown MongoDB URL to match your MongoDB deployment:

mongosh "mongodb+srv://myuser:mypassd@mycluster.a12z.mongodb.net/"
use personsdb;
db.people.find();

In the output above you should see the encrypted values for the fields ssn, mobile and email.

NOTE: If you want to start over again, first drop the created databases csfle and personsdb.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment