Skip to content

Instantly share code, notes, and snippets.

@rohit-lakhanpal

rohit-lakhanpal/graphrag-accelerator-cheat-sheet.md

Last active July 23, 2024 05:06
Show Gist options
  • Save rohit-lakhanpal/1cea160ffe0de4cbcb52f2046ebdfd00 to your computer and use it in GitHub Desktop.
Save rohit-lakhanpal/1cea160ffe0de4cbcb52f2046ebdfd00 to your computer and use it in GitHub Desktop.
Download ZIP
This guide provides step-by-step instructions to set up the graphrag-accelerator project. Follow each section carefully to ensure a smooth setup process.
Raw
graphrag-accelerator-cheat-sheet.md

Graphrag-Accelerator Setup Guide

This guide provides step-by-step instructions to set up the graphrag-accelerator project. Follow each section carefully to ensure a smooth setup process.

Table of Contents

  1. Clone the Repository
  2. Open in Visual Studio Code
  3. Open in Container
  4. Login to Azure
  5. Register Providers
  6. Deploy Azure OpenAI Services
  7. Create Resource Group for Graphrag-Accelerator
  8. Update Deployment Parameters
  9. Modify VM Sizes
  10. Modify the deploy.sh script to fix errors
  11. Deploy Solution Accelerator

Clone the Repository

Clone the graphrag-accelerator repository and navigate to the directory.

git clone https://github.com/Azure-Samples/graphrag-accelerator.git
cd graphrag-accelerator

Open in Visual Studio Code

Open the repository in Visual Studio Code.

code .

Open in Container

Use the Visual Studio Code Remote - Containers extension to open the repository in a container, ensuring all dependencies are installed in the container.

Login to Azure

Login to your Azure account.

az login

Register Providers

Register the necessary Azure providers.

az provider register --namespace Microsoft.OperationsManagement
az provider register --namespace Microsoft.AlertsManagement

Verify that the providers are registered.

az provider show --namespace Microsoft.OperationsManagement -o table
az provider show --namespace Microsoft.AlertsManagement -o table

Note: The registration process can take a few minutes. Do not proceed until the RegistrationState is Registered.

Deploy Azure OpenAI Services

Deploy an Azure OpenAI Service in a Separate Resource Group

Set up and deploy the Azure OpenAI service.

# Set a resource group name
oai_region="eastus2"
oai_prefix="shared-oai"
oai_rg="rg-${oai_prefix}-${oai_region}"

# Add a random number to the service name
i=$((RANDOM%99999+11111))

# Create a resource group in eastus2
az group create --name "$oai_rg" --location eastus2

# Create an Azure OpenAI service in the resource group
oai_svc="${oai_prefix}-svc-$i"
az cognitiveservices account create \
  --name "$oai_svc" \
  --resource-group "$oai_rg" \
  --kind OpenAI \
  --sku S0 \
  --location "$oai_region" \
  --yes

Deploy Models

Deploy various models to the Azure OpenAI service.

# Deploy gpt-4o model
az cognitiveservices account deployment create \
  --name "$oai_svc" \
  --resource-group "$oai_rg" \
  --deployment-name "gpt-4o" \
  --model-name "gpt-4o" \
  --model-version "2024-05-13" \
  --sku-capacity 1 \
  --model-format OpenAI 

# Deploy text-embedding-3-large model
az cognitiveservices account deployment create \
  --name "$oai_svc" \
  --resource-group "$oai_rg" \
  --deployment-name "text-embedding-3-large" \
  --model-name "text-embedding-3-large" \
  --model-version "1" \
  --sku-capacity 1 \
  --model-format OpenAI

# Deploy text-embedding-3-small model
az cognitiveservices account deployment create \
  --name "$oai_svc" \
  --resource-group "$oai_rg" \
  --deployment-name "text-embedding-3-small" \
  --model-name "text-embedding-3-small" \
  --model-version "1" \
  --sku-capacity 1 \
  --model-format OpenAI

# Deploy text-embedding-ada-002 model
az cognitiveservices account deployment create \
  --name "$oai_svc" \
  --resource-group "$oai_rg" \
  --deployment-name "text-embedding-ada-002" \
  --model-name "text-embedding-ada-002" \
  --model-version "2" \
  --sku-capacity 1 \
  --model-format OpenAI

Note: If concerned about throttling, consider editing the deployment in the OAI Portal (https://oai.azure.com) to maximize the Tokens per Minute Rate Limit (thousands) and enable dynamic quotas if available.

Ensure that the Azure OpenAI Service allows local auth access

Ensure that the Azure OpenAI service allows local auth access. (This is not recommended for production environments.)

oai_setting=$( az cognitiveservices account list --query "[?contains(name, '$GRAPHRAG_API_BASE')] | [0].properties.disableLocalAuth" -o json)
if [ "$oai_setting" == "true" ]; then
    echo "Enabling local auth access for $oai_svc"    
    az resource update --ids $(az cognitiveservices account show --name "$oai_svc" --resource-group "$oai_rg" --query id -o tsv) --set properties.disableLocalAuth=false

    # Wait for the change to take effect
    sleep 10

    # Verify that local auth access is enabled
    oai_setting=$( az cognitiveservices account list --query "[?contains(name, '$GRAPHRAG_API_BASE')] | [0].properties.disableLocalAuth" -o json)
    if [ "$oai_setting" == "false" ]; then
        echo "Local auth access enabled for $oai_svc"
    else
        echo "Failed to enable local auth access for $oai_svc"
    fi

fi

Create Resource Group for the Graphrag-Accelerator

Create a resource group for the graphrag-accelerator.

rg="rg-graphrag-accelerator"
region="eastus2"

az group create --name "$rg" --location "$region"

Update Deployment Parameters

Update the parameters in infra/main.parameters.json.

Create a Copy of the Parameters File

cp infra/deploy.parameters.json infra/main.parameters.json

Set Parameters

Set various parameters in the infra/main.parameters.json file.

# Set RESOURCE_GROUP
jq --arg key "RESOURCE_GROUP" --arg value "$rg" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

# Set LOCATION
jq --arg key "LOCATION" --arg value "$region" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

# Set GRAPHRAG_API_BASE (this will likely change, see the section below)
graphrag_api_base="https://$oai_svc.openai.azure.com"
jq --arg key "GRAPHRAG_API_BASE" --arg value "$graphrag_api_base" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

# Set GRAPHRAG_API_VERSION
graphrag_api_version="2023-03-15-preview"
jq --arg key "GRAPHRAG_API_VERSION" --arg value "$graphrag_api_version" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

# Set GRAPHRAG_LLM_DEPLOYMENT_NAME and GRAPHRAG_LLM_MODEL
jq --arg key "GRAPHRAG_LLM_DEPLOYMENT_NAME" --arg value "gpt-4o" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json
jq --arg key "GRAPHRAG_LLM_MODEL" --arg value "gpt-4o" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

# Set GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME and GRAPHRAG_EMBEDDING_MODEL
jq --arg key "GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME" --arg value "text-embedding-3-large" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json
jq --arg key "GRAPHRAG_EMBEDDING_MODEL" --arg value "text-embedding-3-large" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

Modify VM Sizes

  1. Open infra/core/aks/aks.bicep and look for systemVMSize. It is currently set to standard_d4s_v5.

  2. Now also look for graphragVMSize. It iscurrently set to standard_e16as_v5.

  3. It is likely that when you run the script, you might get an error that the size is not available. This is what I got:

    The VM size of standard_e16as_v5 is not allowed in your subscription in location 'eastus2'. 

The available VM sizes are 'standard_a2_v2,standard_a2m_v2,standard_a4_v2,standard_a4m_v2,standard_a8_v2,standard_a8m_v2,standard_b12ms,standard_b16als_v2,standard_b16as_v2,standard_b16ls_v2,standard_b16ms,standard_b16pls_v2,standard_b16ps_v2,standard_b16s_v2,standard_b20ms,standard_b2als_v2,standard_b2as_v2,standard_b2ats_v2,standard_b2ls_v2,standard_b2ms,standard_b2pls_v2,standard_b2ps_v2,standard_b2pts_v2,standard_b2s,standard_b2s_v2,standard_b2ts_v2,standard_b32als_v2,standard_b32as_v2,standard_b32ls_v2,standard_b32s_v2,standard_b4als_v2,standard_b4as_v2,standard_b4ls_v2,standard_b4ms,standard_b4pls_v2,standard_b4ps_v2,standard_b4s_v2,standard_b8als_v2,standard_b8as_v2,standard_b8ls_v2,standard_b8ms,standard_b8pls_v2,standard_b8ps_v2,standard_b8s_v2,standard_d11,standard_d11_v2,standard_d12,standard_d12_v2,standard_d13,standard_d13_v2,standard_d14,standard_d14_v2,standard_d16_v3,standard_d16_v4,standard_d16_v5,standard_d16d_v4,standard_d16d_v5,standard_d16ds_v4,standard_d16ds_v5,standard_d16lds_v5,standard_d16ls_v5,standard_d16pds_v5,standard_d16plds_v5,standard_d16pls_v5,standard_d16ps_v5,standard_d16s_v3,standard_d16s_v4,standard_d16s_v5,standard_d2,standard_d2_v2,standard_d2_v3,standard_d2_v4,standard_d2_v5,standard_d2d_v4,standard_d2d_v5,standard_d2ds_v4,standard_d2ds_v5,standard_d2lds_v5,standard_d2ls_v5,standard_d2pds_v5,standard_d2plds_v5,standard_d2pls_v5,standard_d2ps_v5,standard_d2s_v3,standard_d2s_v4,standard_d2s_v5,standard_d3,standard_d32_v3,standard_d32_v4,standard_d32_v5,standard_d32d_v4,standard_d32d_v5,standard_d32ds_v4,standard_d32ds_v5,standard_d32lds_v5,standard_d32ls_v5,standard_d32pds_v5,standard_d32plds_v5,standard_d32pls_v5,standard_d32ps_v5,standard_d32s_v3,standard_d32s_v4,standard_d32s_v5,standard_d3_v2,standard_d4,standard_d48_v3,standard_d48_v4,standard_d48_v5,standard_d48d_v4,standard_d48d_v5,standard_d48ds_v4,standard_d48ds_v5,standard_d48lds_v5,standard_d48ls_v5,standard_d48pds_v5,standard_d48plds_v5,standard_d48pls_v5,standard_d48ps_v5,standard_d48s_v3,standard_d48s_v4,standard_d48s_v5,standard_d4_v2,standard_d4_v3,standard_d4_v4,standard_d4_v5,standard_d4d_v4,standard_d4d_v5,standard_d4ds_v4,standard_d4ds_v5,standard_d4lds_v5,standard_d4ls_v5,standard_d4pds_v5,standard_d4plds_v5,standard_d4pls_v5,standard_d4ps_v5,standard_d4s_v3,standard_d4s_v4,standard_d4s_v5,standard_d5_v2,standard_d64_v3,standard_d64_v4,standard_d64_v5,standard_d64d_v4,standard_d64d_v5,standard_d64ds_v4,standard_d64ds_v5,standard_d64lds_v5,standard_d64ls_v5,standard_d64pds_v5,standard_d64plds_v5,standard_d64pls_v5,standard_d64ps_v5,standard_d64s_v3,standard_d64s_v4,standard_d64s_v5,standard_d8_v3,standard_d8_v4,standard_d8_v5,standard_d8d_v4,standard_d8d_v5,standard_d8ds_v4,standard_d8ds_v5,standard_d8lds_v5,standard_d8ls_v5,standard_d8pds_v5,standard_d8plds_v5,standard_d8pls_v5,standard_d8ps_v5,standard_d8s_v3,standard_d8s_v4,standard_d8s_v5,standard_d96_v5,standard_d96d_v5,standard_d96ds_v5,standard_d96lds_v5,standard_d96ls_v5,standard_d96s_v5,standard_dc16ds_v3,standard_dc16eds_v5,standard_dc16es_v5,standard_dc16s_v3,standard_dc24ds_v3,standard_dc24s_v3,standard_dc2ds_v3,standard_dc2eds_v5,standard_dc2es_v5,standard_dc2s_v3,standard_dc32ds_v3,standard_dc32eds_v5,standard_dc32es_v5,standard_dc32s_v3,standard_dc48ds_v3,standard_dc48eds_v5,standard_dc48es_v5,standard_dc48s_v3,standard_dc4ds_v3,standard_dc4eds_v5,standard_dc4es_v5,standard_dc4s_v3,standard_dc64eds_v5,standard_dc64es_v5,standard_dc8ds_v3,standard_dc8eds_v5,standard_dc8es_v5,standard_dc8s_v3,standard_dc96eds_v5,standard_dc96es_v5,standard_ds11,standard_ds11-1_v2,standard_ds11_v2,standard_ds12,standard_ds12-1_v2,standard_ds12-2_v2,standard_ds12_v2,standard_ds13,standard_ds13-2_v2,standard_ds13-4_v2,standard_ds13_v2,standard_ds14,standard_ds14-4_v2,standard_ds14-8_v2,standard_ds14_v2,standard_ds2,standard_ds2_v2,standard_ds3,standard_ds3_v2,standard_ds4,standard_ds4_v2,standard_ds5_v2,standard_e104i_v5,standard_e104id_v5,standard_e104ids_v5,standard_e104is_v5,standard_e112iads_v5,standard_e112ias_v5,standard_e112ibds_v5,standard_e112ibs_v5,standard_e16-4ds_v4,standard_e16-4ds_v5,standard_e16-4s_v3,standard_e16-4s_v4,standard_e16-4s_v5,standard_e16-8ds_v4,standard_e16-8ds_v5,standard_e16-8s_v3,standard_e16-8s_v4,standard_e16-8s_v5,standard_e16_v3,standard_e16_v4,standard_e16_v5,standard_e16bds_v5,standard_e16bs_v5,standard_e16d_v4,standard_e16d_v5,standard_e16ds_v4,standard_e16ds_v5,standard_e16pds_v5,standard_e16ps_v5,standard_e16s_v3,standard_e16s_v4,standard_e16s_v5,standard_e20_v3,standard_e20_v4,standard_e20_v5,standard_e20d_v4,standard_e20d_v5,standard_e20ds_v4,standard_e20ds_v5,standard_e20pds_v5,standard_e20ps_v5,standard_e20s_v3,standard_e20s_v4,standard_e20s_v5,standard_e2_v3,standard_e2_v4,standard_e2_v5,standard_e2bds_v5,standard_e2bs_v5,standard_e2d_v4,standard_e2d_v5,standard_e2ds_v4,standard_e2ds_v5,standard_e2pds_v5,standard_e2ps_v5,standard_e2s_v3,standard_e2s_v4,standard_e2s_v5,standard_e32-16ds_v4,standard_e32-16ds_v5,standard_e32-16s_v3,standard_e32-16s_v4,standard_e32-16s_v5,standard_e32-8ds_v4,standard_e32-8ds_v5,standard_e32-8s_v3,standard_e32-8s_v4,standard_e32-8s_v5,standard_e32_v3,standard_e32_v4,standard_e32_v5,standard_e32bds_v5,standard_e32bs_v5,standard_e32d_v4,standard_e32d_v5,standard_e32ds_v4,standard_e32ds_v5,standard_e32pds_v5,standard_e32ps_v5,standard_e32s_v3,standard_e32s_v4,standard_e32s_v5,standard_e4-2ds_v4,standard_e4-2ds_v5,standard_e4-2s_v3,standard_e4-2s_v4,standard_e4-2s_v5,standard_e48_v3,standard_e48_v4,standard_e48_v5,standard_e48bds_v5,standard_e48bs_v5,standard_e48d_v4,standard_e48d_v5,standard_e48ds_v4,standard_e48ds_v5,standard_e48s_v3,standard_e48s_v4,standard_e48s_v5,standard_e4_v3,standard_e4_v4,standard_e4_v5,standard_e4bds_v5,standard_e4bs_v5,standard_e4d_v4,standard_e4d_v5,standard_e4ds_v4,standard_e4ds_v5,standard_e4pds_v5,standard_e4ps_v5,standard_e4s_v3,standard_e4s_v4,standard_e4s_v5,standard_e64-16ds_v4,standard_e64-16ds_v5,standard_e64-16s_v3,standard_e64-16s_v4,standard_e64-16s_v5,standard_e64-32ds_v4,standard_e64-32ds_v5,standard_e64-32s_v3,standard_e64-32s_v4,standard_e64-32s_v5,standard_e64_v3,standard_e64_v4,standard_e64_v5,standard_e64bds_v5,standard_e64bs_v5,standard_e64d_v4,standard_e64d_v5,standard_e64ds_v4,standard_e64ds_v5,standard_e64i_v3,standard_e64is_v3,standard_e64s_v3,standard_e64s_v4,standard_e64s_v5,standard_e8-2ds_v4,standard_e8-2ds_v5,standard_e8-2s_v3,standard_e8-2s_v4,standard_e8-2s_v5,standard_e8-4ds_v4,standard_e8-4ds_v5,standard_e8-4s_v3,standard_e8-4s_v4,standard_e8-4s_v5,standard_e80ids_v4,standard_e80is_v4,standard_e8_v3,standard_e8_v4,standard_e8_v5,standard_e8bds_v5,standard_e8bs_v5,standard_e8d_v4,standard_e8d_v5,standard_e8ds_v4,standard_e8ds_v5,standard_e8pds_v5,standard_e8ps_v5,standard_e8s_v3,standard_e8s_v4,standard_e8s_v5,standard_e96-24ds_v5,standard_e96-24s_v5,standard_e96-48ds_v5,standard_e96-48s_v5,standard_e96_v5,standard_e96bds_v5,standard_e96bs_v5,standard_e96d_v5,standard_e96ds_v5,standard_e96s_v5,standard_ec128eds_v5,standard_ec128es_v5,standard_ec128ieds_v5,standard_ec128ies_v5,standard_ec16eds_v5,standard_ec16es_v5,standard_ec2eds_v5,standard_ec2es_v5,standard_ec32eds_v5,standard_ec32es_v5,standard_ec48eds_v5,standard_ec48es_v5,standard_ec4eds_v5,standard_ec4es_v5,standard_ec64eds_v5,standard_ec64es_v5,standard_ec8eds_v5,standard_ec8es_v5,standard_f16,standard_f16s,standard_f16s_v2,standard_f2,standard_f2s,standard_f2s_v2,standard_f32s_v2,standard_f4,standard_f48s_v2,standard_f4s,standard_f4s_v2,standard_f64s_v2,standard_f72s_v2,standard_f8,standard_f8s,standard_f8s_v2,standard_hc44-16rs,standard_hc44-32rs,standard_hc44rs,standard_l16s_v3,standard_l32s_v3,standard_l48s_v3,standard_l64s_v3,standard_l80s_v3,standard_l8s_v3,standard_m12ds_v3,standard_m12s_v3,standard_m176ds_3_v3,standard_m176ds_4_v3,standard_m176s_3_v3,standard_m176s_4_v3,standard_m24ds_v3,standard_m24s_v3,standard_m48ds_1_v3,standard_m48s_1_v3,standard_m96ds_1_v3,standard_m96ds_2_v3,standard_m96s_1_v3,standard_m96s_2_v3,standard_nc24ads_a100_v4,standard_nc40ads_h100_v5,standard_nc48ads_a100_v4,standard_nc80adis_h100_v5,standard_nc96ads_a100_v4,standard_nv12s_v2,standard_nv24s_v2,standard_nv6s_v2

    To remedy this error, I changed the graphragVMSize from standard_e16as_v5 to standard_e16_v5 (smaller machines yeilded errors).

Modify the deploy.sh script to fix errors

I'm in the process of creating a PR for this. But in the meanwhile, you can modify the infra/deploy.sh script.

Find the function called assignAOAIRoleToManagedIdentity. See this link for the code.

Here the problem I faced was with a new Azure OpenAI service, the properties.endpoint was not in the https://<my_openai_name>.openai.azure.com format. Instead, it was in the https://<my_openai_name>.cognitiveservices.azure.com format. This caused a problem when the following command was run:

local scope=$(az cognitiveservices account list --query "[?contains(properties.endpoint, '$GRAPHRAG_API_BASE')] | [0].id" -o json)

To fix this, I modified the code to look for the properties.endpoint in the https://<my_openai_name>.cognitiveservices.azure.com format.

echo "Retrieving Cognitive Services account id..."
echo "GraphRAG API Base: $GRAPHRAG_API_BASE"

# local scope=$(az cognitiveservices account list --query "[?contains(properties.endpoint, '$GRAPHRAG_API_BASE')] | [0].id" -o json)
local scope=$(az cognitiveservices account list --query "[?contains(name, '$GRAPHRAG_API_BASE')] | [0].id" -o json)
echo "Initial Scope: $scope" # debugging purposes
scope=$(jq -r <<< $scope) # strip out quotes
echo "Final Scope: $scope" # debugging purposes

Note: For this to work, I had to ensure that the GRAPHRAG_API_BASE env variable was NOT in the https://<my_openai_name>.cognitiveservices.azure.com format BUT INSTEAD WAS JUST THE NAME OF THE OPENAI SERVICE. To make this work, now change the GRAPHRAG_API_BASE in the infra/main.parameters.json file to just the name of the OpenAI service.

# graphrag_api_base="https://$oai_svc.openai.azure.com" # change this
graphrag_api_base=$oai_svc # Now instead of the endpoint we pass in the name of the OpenAI service
jq --arg key "GRAPHRAG_API_BASE" --arg value "$graphrag_api_base" '.[$key] = $value' infra/main.parameters.json > infra/main.parameters.json.tmp && mv infra/main.parameters.json.tmp infra/main.parameters.json

Now is a good time to manually validate the paramters in the infra/main.parameters.json file. Mine looks something like this ...

{
    "GRAPHRAG_API_BASE": "shared-oai-svc-56521",
    "GRAPHRAG_API_VERSION": "2024-06-01",
    "GRAPHRAG_EMBEDDING_DEPLOYMENT_NAME": "text-embedding-3-large",
    "GRAPHRAG_EMBEDDING_MODEL": "text-embedding-3-large",
    "GRAPHRAG_LLM_DEPLOYMENT_NAME": "gpt-4o",
    "GRAPHRAG_LLM_MODEL": "gpt-4o",
    "LOCATION": "eastus2",
    "RESOURCE_GROUP": "rg-graphrag-accelerator"
}

Deploy Solution Accelerator

Deploy the solution accelerator to the resource group.

cd infra
chmod 705
bash deploy.sh -p main.parameters.json -g

IMPORTANT: The -g option is for developer use only. It grants deployer access to Azure Storage, AI Search, and CosmosDB, disables private endpoints, and enables debug mode. The initial deployment might take ~40-50 minutes. Subsequent deployments will be faster.

Follow these steps to set up the graphrag-accelerator project successfully. If you encounter any issues, refer to the Azure documentation or seek support from the community.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment