{"id":1716,"date":"2023-07-08T20:11:52","date_gmt":"2023-07-09T03:11:52","guid":{"rendered":"http:\/\/blog.nillsf.com\/?p=1716"},"modified":"2023-07-08T20:11:55","modified_gmt":"2023-07-09T03:11:55","slug":"how-to-configure-langchain-to-use-azure-openai-in-python","status":"publish","type":"post","link":"https:\/\/blog.nillsf.com\/index.php\/2023\/07\/08\/how-to-configure-langchain-to-use-azure-openai-in-python\/","title":{"rendered":"How to configure LangChain to use Azure OpenAI in Python"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">I spent some time last week running sample apps using LangChain to interact with Azure OpenAI. Most (if not all) of the examples connect to OpenAI natively, and not to Azure OpenAI. It took a little bit of tinkering on my end to get LangChain to connect to Azure OpenAI; so, I decided to write down my thoughts about you can use LangChain to connect to Azure OpenAI.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">LangChain is a framework designed to simplify the creation of applications using large language models (LLMs).&nbsp;It is a language model integration framework that can be used for document analysis and summarization, chatbots, and code analysis.&nbsp;It allows you to build more complicated interactions with a language model without having to write a lot of the prompt engineering and orchestration.&nbsp;<a href=\"https:\/\/docs.langchain.com\/docs\/\">You can learn more about LangChain by visiting their official website<\/a>. <\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What you need from Azure OpenAI<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I&#8217;ll assume if you&#8217;re reading this, that you already have access to Azure OpenAI. If you don&#8217;t have access already, please refer <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/cognitive-services\/openai\/overview#how-do-i-get-access-to-azure-openai\">to the Azure documentation<\/a> for how to get access.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">We&#8217;ll need to get the following information from the Azure OpenAI service:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>\nOPENAI_API_BASE = \"https:\/\/nillsf-openai.openai.azure.com\/\"\nOPENAI_API_KEY = \"xxx\"\nOPENAI_API_TYPE = \"azure\"\nOPENAI_DEPLOYMENT_NAME = \"nillsf-embeddings\"\nOPENAI_DEPLOYMENT_VERSION = \"2\"\nOPENAI_MODEL_NAME=\"text-embedding-ada-002\"<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The first two items you can get from the <a href=\"https:\/\/portal.azure.com\/#view\/Microsoft_Azure_ProjectOxford\/CognitiveServicesHub\/~\/OpenAI\">Azure portal<\/a>. Open your OpenAI resource, and select &#8220;Keys and Endpoint&#8221; in the left-hand navigation. There you&#8217;ll find your endpoint and the two keys. Grab one of the keys, you don&#8217;t need both.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">(side-note: you can optionally configure Azure OpenAI and LangChain to leverage AAD authentication. For this post, we&#8217;ll assume you&#8217;ll use key based authentication.)<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"601\" src=\"\/wp-content\/uploads\/2023\/07\/image-1024x601.png\" alt=\"\" class=\"wp-image-1717\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-1024x601.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-300x176.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-768x451.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image.png 1067w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption>Leverage the Azure portal to get an API key and the API endpoint<\/figcaption><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Next, you&#8217;ll want to head over to the <a href=\"https:\/\/oai.azure.com\/\">OpenAI studio<\/a> to get the deployment information. Please take care of noting the differences between the deployment name and the model name &#8211; in case they&#8217;re different.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"393\" src=\"\/wp-content\/uploads\/2023\/07\/image-2-1024x393.png\" alt=\"\" class=\"wp-image-1719\" srcset=\"https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-2-1024x393.png 1024w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-2-300x115.png 300w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-2-768x295.png 768w, https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/image-2.png 1371w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><figcaption> Leverage Azure OpenAI studio to get the deployment name, model name and model version. <\/figcaption><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">How to load the settings file using dotenv<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">It&#8217;s common in most LangChain examples I&#8217;ve seen to leverage dotenv to load environment variables. Let&#8217;s follow that practice here as well.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Save the file you created earlier (repeated below) with your OpenAI config information as a file called <code>.env<\/code>.  In your Python code, you can then load those variables as environment variables using <code>dotenv.load_dotenv()<\/code>. <\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>OPENAI_API_BASE = \"https:\/\/nillsf-openai.openai.azure.com\/\"\nOPENAI_API_KEY = \"xxx\"\nOPENAI_API_TYPE = \"azure\"\nOPENAI_DEPLOYMENT_NAME = \"nillsf-embeddings\"\nOPENAI_DEPLOYMENT_VERSION = \"2\"\nOPENAI_MODEL_NAME=\"text-embedding-ada-002\"<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">How to connect LangChain to Azure OpenAI<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There&#8217;s a couple of OpenAI models available in LangChain. In what follows, we&#8217;ll cover two examples, which I hope is enough to get you started and pointed in the right direction:<\/p>\n\n\n\n<ul class=\"wp-block-list\"><li>Embeddings<\/li><li>GPT-3.5-turbo (chat)<\/li><\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Each works very similarly in Azure OpenAI setup, and each has a couple of ways to configure the Azure endpoint. Let&#8217;s cover them:<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Embeddings<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">By only pointing to the deployment name<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If you followed the instructions up to this point and specified the <code>.env<\/code> file mentioned earlier &#8211; you only need to s<a href=\"https:\/\/github.com\/NillsF\/blog\/blob\/master\/langchain-azure-openai\/0-model-name-only.py\">pecify the deployment_name in the OpenAIEmbeddings<\/a> rather than the model name you&#8217;d use if you were OpenAI in the non-Azure version:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import os\nfrom langchain.embeddings import OpenAIEmbeddings\nimport dotenv\n\n# Load environment variables from .env file\ndotenv.load_dotenv()\n\n# Create an instance of the OpenAIEmbeddings class using Azure OpenAI\nembeddings = OpenAIEmbeddings(\n    deployment=os.getenv(\"OPENAI_DEPLOYMENT_NAME\"),\n    chunk_size=1)\n\n# Testing embeddings\ntxt = \"This is how you configure it directly in the constructor.\"\n\n# Embed a single document\ne = embeddings.embed_query(txt)\n\nprint(len(e)) # should be 1536<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Constructor method<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If for any reason you either don&#8217;t feel like using the .env file or want to more specifically control the environment you use, you can do that as well. For the LangChain OpenAI embeddings models, it&#8217;s possible to specify all the <a href=\"https:\/\/github.com\/NillsF\/blog\/blob\/master\/langchain-azure-openai\/1-constructor.py\">Azure endpoints in the constructor of the model in Pytho<\/a>n:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>embeddings = OpenAIEmbeddings(\n    openai_api_type=\"azure\",\n    openai_api_key=os.getenv(\"OPENAI_API_KEY\"),\n    openai_api_base=os.getenv(\"OPENAI_API_BASE\"),\n    deployment=os.getenv(\"OPENAI_DEPLOYMENT_NAME\"),\n    model=os.getenv(\"OPENAI_MODEL_NAME\"),\n    chunk_size=1)<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">In this case, I&#8217;m still getting the variables from the <code>.env<\/code> file (<code>os.getenv()<\/code> method) &#8212; but you could also go ahead and configure those any other way.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Side note: Don&#8217;t bother importing and configuring openai package<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Another approach I&#8217;ve seen is to import and configure the openai package in Python, rather than deal with the environment variables. Some code samples online, follow this approach that doesn&#8217;t work:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># This does nothing for OpenAIEmbeddings()\nimport openai\nopenai.api_type=\"azure\"\nopenai.api_key=\"xxx\"\nopenai.api_base=\"https:\/\/nillsf-openai.openai.azure.com\/\"\nopenai.api_version = \"2023-05-15\"<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Chat aka gpt-35-turbo<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Chat works a bit different from embeddings. With embeddings, as you might have noticed; you can import the same class (nothing Azure specific about it), but for chat; you need to import a specific class (<code>AzureChatOpenAI <\/code>). Let&#8217;s have a look.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">First, make a small change to <a href=\"https:\/\/github.com\/NillsF\/blog\/blob\/master\/langchain-azure-openai\/chat.env\">your .env file<\/a> (which I named <code>chat.env<\/code> for the second set of examples) to reflect the name of the gpt-35-turbo model:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>OPENAI_API_BASE = \"https:\/\/nillsf-openai.openai.azure.com\/\"\nOPENAI_API_KEY = \"xxx\"\nOPENAI_API_TYPE = \"azure\"\nOPENAI_DEPLOYMENT_NAME = \"nillsfgpt35turbo\"\nOPENAI_DEPLOYMENT_VERSION = \"0301\"\nOPENAI_MODEL_NAME=\"gpt-35-turbo\"<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Let&#8217;s now look at how to use this:<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Environment variables with deployment name<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\"><a href=\"https:\/\/github.com\/NillsF\/blog\/blob\/master\/langchain-azure-openai\/2-chat-model-name.py\">Pretty similar to the previous example actually<\/a>, we load the environment variables and pass the deployment name of our GPT 3.5 model into the constructor:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import os\nimport dotenv\n\nfrom langchain.chat_models import AzureChatOpenAI\nfrom langchain.schema import HumanMessage\n\n# Load environment variables from .env file\ndotenv.load_dotenv(dotenv_path='.\/chat.env')\n\n# Create an instance of the AzureChatOpenAI class using Azure OpenAI\nllm = AzureChatOpenAI(\n    deployment_name=os.getenv(\"OPENAI_DEPLOYMENT_NAME\"),\n    temperature=0.7,\n    openai_api_version=\"2023-05-15\")\n\n# Testing chat llm  \nres = llm(&#91;HumanMessage(content=\"Tell me a joke about a penguin sitting on a fridge.\")])\nprint(res)\n<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">Constructor method<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Again, similar to the previous example with embeddings; you can be more specific with your models or endpoints in case you want to:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>llm = AzureChatOpenAI(\n    openai_api_type=\"azure\",\n    openai_api_key=os.getenv(\"OPENAI_API_KEY\"),\n    openai_api_base=os.getenv(\"OPENAI_API_BASE\"),\n    deployment_name=os.getenv(\"OPENAI_DEPLOYMENT_NAME\"),\n    model=os.getenv(\"OPENAI_MODEL_NAME\"),\n    temperature=0.7,\n    openai_api_version=\"2023-05-15\")<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">That&#8217;s it! You now have the necessary information to connect LangChain to Azure OpenAI for both embeddings and the chat model (GPT-3.5-turbo).<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Feel free to explore further and customize the integration according to your specific needs. LangChain provides a powerful framework for leveraging language models and allows you to build fully featured AI apps.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Remember to refer to the official documentation of LangChain and Azure OpenAI for more detailed guidance and additional features.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I spent some time last week running sample apps using LangChain to interact with Azure OpenAI. Most (if not all) of the examples connect to OpenAI natively, and not to Azure OpenAI. It took a little bit of tinkering on my end to get LangChain to connect to Azure OpenAI; so, I decided to write [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":1727,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[2,47,190,31],"tags":[192,191],"class_list":["post-1716","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-data-science","category-openai","category-software-development","tag-langchain","tag-openai"],"jetpack_featured_media_url":"https:\/\/nillsfblog.blob.core.windows.net\/media\/2023\/07\/langchain-azure-open.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1716","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/comments?post=1716"}],"version-history":[{"count":5,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1716\/revisions"}],"predecessor-version":[{"id":1726,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1716\/revisions\/1726"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/media\/1727"}],"wp:attachment":[{"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/media?parent=1716"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/categories?post=1716"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.nillsf.com\/index.php\/wp-json\/wp\/v2\/tags?post=1716"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}