Skip to main content

Oracle Cloud Infrastructure (OCI)

LiteLLM supports the following models for OCI on-demand GenAI API.

Check the OCI Models List to see if the model is available for your region.

Supported Models​

Meta Llama Models​

  • meta.llama-4-maverick-17b-128e-instruct-fp8
  • meta.llama-4-scout-17b-16e-instruct
  • meta.llama-3.3-70b-instruct
  • meta.llama-3.2-90b-vision-instruct
  • meta.llama-3.1-405b-instruct

xAI Grok Models​

  • xai.grok-4
  • xai.grok-3
  • xai.grok-3-fast
  • xai.grok-3-mini
  • xai.grok-3-mini-fast

Cohere Models​

  • cohere.command-latest
  • cohere.command-a-03-2025
  • cohere.command-plus-latest

Authentication​

LiteLLM uses OCI signing key authentication. Follow the official Oracle tutorial to create a signing key and obtain the following parameters:

  • user
  • fingerprint
  • tenancy
  • region
  • key_file

Usage​

Input the parameters obtained from the OCI signing key creation process into the completion function.

import os
from litellm import completion

messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(
    model="oci/xai.grok-4",
    messages=messages,
    oci_region=<your_oci_region>,
    oci_user=<your_oci_user>,
    oci_fingerprint=<your_oci_fingerprint>,
    oci_tenancy=<your_oci_tenancy>,
    oci_serving_mode="ON_DEMAND",  # Optional, default is "ON_DEMAND". Other option is "DEDICATED"
    # Provide either the private key string OR the path to the key file:
    # Option 1: pass the private key as a string
    oci_key=<string_with_content_of_oci_key>,
    # Option 2: pass the private key file path
    # oci_key_file="<path/to/oci_key.pem>",
    oci_compartment_id=<oci_compartment_id>,
)
print(response)

Usage - Streaming​

Just set stream=True when calling completion.

import os
from litellm import completion

messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(
    model="oci/xai.grok-4",
    messages=messages,
    stream=True,
    oci_region=<your_oci_region>,
    oci_user=<your_oci_user>,
    oci_fingerprint=<your_oci_fingerprint>,
    oci_tenancy=<your_oci_tenancy>,
    oci_serving_mode="ON_DEMAND",  # Optional, default is "ON_DEMAND". Other option is "DEDICATED"
    # Provide either the private key string OR the path to the key file:
    # Option 1: pass the private key as a string
    oci_key=<string_with_content_of_oci_key>,
    # Option 2: pass the private key file path
    # oci_key_file="<path/to/oci_key.pem>",
    oci_compartment_id=<oci_compartment_id>,
)
for chunk in response:
    print(chunk["choices"][0]["delta"]["content"])  # same as openai format

Usage Examples by Model Type​

Using Cohere Models​

from litellm import completion

messages = [{"role": "user", "content": "Explain quantum computing"}]
response = completion(
    model="oci/cohere.command-latest",
    messages=messages,
    oci_region="us-chicago-1",
    oci_user=<your_oci_user>,
    oci_fingerprint=<your_oci_fingerprint>,
    oci_tenancy=<your_oci_tenancy>,
    oci_key=<string_with_content_of_oci_key>,
    oci_compartment_id=<oci_compartment_id>,
)
print(response)