Support alternative OpenAI hosts (OpenAI.com, GitHub Models, Ollama)#264
Support alternative OpenAI hosts (OpenAI.com, GitHub Models, Ollama)#264
Conversation
…for chat and embeddings Add OPENAI_CHAT_HOST and OPENAI_EMBED_HOST environment variables to configure which OpenAI-compatible provider to use. Supported values: azure (default), openai, github, ollama. Key changes: - utils.py: embeddings_api() and chat_api() now accept host parameter and return appropriate LangChain client (Azure or non-Azure) - base.py/setup.py: Widen type hints to LangChain base types (BaseChatModel, Embeddings) for provider-agnostic usage - config_base.py: Read OPENAI_CHAT_HOST, OPENAI_EMBED_HOST, and AZURE_OPENAI_EMBEDDINGS_DIMENSIONS env vars - Configurable embedding dimensions for compatibility with models that produce different dimension sizes (e.g. Ollama's nomic-embed-text at 768 dims) - .env.example: Add configuration examples for all supported hosts - README.md: Add documentation section for alternative OpenAI hosts - Tests: Add tests for all host types in test_utilities.py Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/df9650f5-df31-47d2-ad68-3fd4a83f3e1d
Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/df9650f5-df31-47d2-ad68-3fd4a83f3e1d
|
@copilot fix failing ci |
- Use BaseChatOpenAI instead of BaseChatModel for chat type hints (fixes mypy errors for temperature attribute access in rag.py) - Wrap Ollama API key string with SecretStr (fixes mypy arg-type error) - Change str+Enum to StrEnum in schemas.py (fixes ruff UP042) - Widen add_data.py embeddings type from AzureOpenAIEmbeddings to Embeddings - Apply ruff formatter to utils.py Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/773ee6b2-1010-4dbb-b4e0-d79b407acb5f
Fixed in 5eef407. Three CI issues:
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot there is a docs folder create a new markdown file in it with docs, don't add it to main readme |
…enai_hosts.md Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/29c12b14-0f21-44db-9eff-fd96136d8a14
|
@copilot rename env vars OPENAI_CHAT_HOST to CHAT_MODEL_HOST and OPENAI_EMBED_HOST to EMBED_MODEL_HOST and add more docs about usage with links to external documentations like microsoft docs/ github docs / openai docs for people to learn more in |
…DEL_HOST, use os.getenv for endpoints, enhance docs Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/58584972-4401-4218-ba14-662df5d7913a
Done in 6c2b500:
Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
There was a problem hiding this comment.
Pull request overview
Adds multi-provider OpenAI-compatible client support across the app (chat + embeddings) by routing to Azure/OpenAI/GitHub Models/Ollama implementations based on new environment variables, while keeping Azure as the default for backwards compatibility.
Changes:
- Add host-based dispatch in
chat_api()/embeddings_api()and widen internal type hints to provider-agnostic LangChain base types. - Introduce
CHAT_MODEL_HOST/EMBED_MODEL_HOSTand optionalAZURE_OPENAI_EMBEDDINGS_DIMENSIONSconfig wiring. - Update tests, examples, and add documentation for alternative hosts.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/quartapp/approaches/utils.py |
Dispatch chat/embeddings clients by host; adds Ollama/GitHub endpoint env support. |
src/quartapp/config_base.py |
Reads new host env vars and optional embedding dimensions into Setup. |
src/quartapp/approaches/setup.py |
Plumbs host + dimensions through setup; widens types for multi-host compatibility. |
src/quartapp/approaches/base.py |
Widened constructor type hints to provider-agnostic interfaces. |
src/quartapp/approaches/schemas.py |
Switch enums to StrEnum (with fallback). |
src/scripts/add_data.py |
Widen embeddings parameter type to Embeddings. |
tests/test_utilities.py |
Adds unit tests covering each host branch for chat/embeddings factories. |
tests/conftest.py |
Updates env var fixtures and Setup fixture parameters for new host args. |
infra/main.bicep |
Renames host parameters and exports new env vars (but still has Azure/OpenAI.com-only constraints). |
infra/main.parameters.json |
Renames host parameters to match new bicep parameter names. |
docs/alternative_openai_hosts.md |
New documentation for configuring OpenAI.com, GitHub Models, and Ollama. |
.env.example |
Adds example configuration blocks for selecting hosts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot address all of the pr comments, run all checks including tests, linting, formatting before committing |
…ust dimensions parsing, updated bicep allowed values and unconditional model env vars Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/95e6b488-a479-4ba1-abe8-0d2e30a41066
Addressed all PR review comments in c722381:
All checks pass: ruff lint ✅, ruff format ✅, mypy ✅, 68/68 unit tests ✅ Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
@copilot fix all of these issues support all hosts/ all models in bicep too |
…oint override; fix bicep OPENAI_API_VERSION and AZURE_OPENAI_ENDPOINT for mixed hosts Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/bb3f28b0-4ce6-4c26-85ef-abeb755905bc
…ENAI_ENDPOINT for Ollama config Co-authored-by: john0isaac <64026625+john0isaac@users.noreply.github.com> Agent-Logs-Url: https://github.com/Azure-Samples/Cosmic-Food-RAG-app/sessions/bb3f28b0-4ce6-4c26-85ef-abeb755905bc
Fixed all issues in a7109b5 and 4e84641:
All checks pass: ruff lint ✅, ruff format ✅, mypy ✅, 68/68 tests ✅ Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 12 out of 12 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…e main.parameters.json and add data handling in add_data.py
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This pull request introduces support for multiple OpenAI-compatible model providers (Azure OpenAI, OpenAI.com, GitHub Models, and Ollama) for both chat and embeddings, enabling flexible configuration via environment variables. It also updates infrastructure and code to generalize model host selection, adds documentation for alternative hosts, and refactors code to use more generic types and improved enum handling.
fixes #74
The most important changes are:
Multi-provider model host support:
CHAT_MODEL_HOSTandEMBED_MODEL_HOSTenvironment variables, with configuration blocks for Azure, OpenAI.com, GitHub Models, and Ollama in.env.example. Each provider has its own set of environment variables, allowing for flexible provider selection and mixing.docs/alternative_openai_hosts.md) explaining how to configure and use each supported provider, including setup instructions and environment variable details.Infrastructure and deployment updates:
infra/main.bicepandinfra/main.parameters.json. Deprecated and replaced old variables with the new, more flexible naming scheme. [1] [2] [3] [4] [5] [6]Codebase refactoring and type improvements:
EmbeddingsandBaseChatOpenAItypes instead of Azure-specific classes, allowing for easier integration of alternative providers. [1] [2] [3] [4]Enum and compatibility improvements:
src/quartapp/approaches/schemas.pyby introducing aStrEnumfallback for compatibility with Python versions earlier than 3.11, ensuring consistent behavior across environments.