Prompty で GPT4 with Vision を使う方法

prompty_sample.py

from promptflow.core import Prompty
from promptflow.contracts.multimedia import PFBytes

image_urls = [
    "画像にアクセスすることができるURL(SAS付きURLとか)",
]
images = [PFBytes(bytes(image_url, "utf-8"), "image/jpeg", source_url=image_url) for image_url in image_urls]
image_paths = [str(image) for image in images]

f = Prompty.load(source="sample.prompty")
completion = f(
    question="アップロードした画像を説明してください。",
    image_paths=image_paths,
    images=images,
)
print(completion)

sample.prompty

---
model:
  api: chat
  configuration:
    type: azure_openai
    connection: default_openai_connection
    azure_deployment: gpt-4o
  parameters:
    temperature: 0
    max_tokens: 4096
inputs:
  question:
    type: string
  images:
    type: list
  image_paths:
    type: list
---

system:
ユーザからの質問に回答してください。

user:
{{question}}
{% for image_path in image_paths %}
{{image_path}}
{% endfor %}