In my two previous blog posts Talking to Oracle Database in plain English and Talking to Oracle Database, this time in plain Slovenian I have been playing with Select AI in Oracle 23ai database.
In these two blog posts I tested how Oracle 23ai feature called Select AI provides SQL access to generative AI using Large Language Models to generate SQL query which is then executed in database.
In this blog post I am testing an option to use Select AI for Retrieval-Augmented Generation (RAG).
Select AI with RAG augments natural language prompt by retrieving data (documents) from vector store (stored in Oracle 23ai). With this additional content, hallucinations can be reduced and much more accurate answers could be retrieved.
Setting it up
Select AI is using Oracle 23ai AI Vector Search for similarity search using vector embeddings. To set the environment for Select AI with RAG two main tasks needs to be performed:
- set up vector store in Object Storage and
- create vector index.
I have set my vector store in OCI Object Storage. For this I am using rag-mini-wikipedia dataset I found on Hugging Face datasets page. Dataset consist of 100+ text files which I uploaded to my Object Storage.
In my example, I am using OpenAI for my LLM. Oracle allows other LLMs as well, and process for setting them up is practically identical.
First, we need to make sure that database allows HTTP connection to OpenAI API endpoint. For example, I am using database user OML_USER to access api.open.com.
Database administrator (ADMIN) needs to run the following code (i.e. SQL Developer):
DBMS_NETWORK_ACL_ADMIN.APPEND_HOST_ACE(
host => 'api.openai.com',
ace => xs$ace_type(privilege_list => xs$name_list('http'),
principal_name => 'OML_USER',
principal_type => xs_acl.ptype_db)
);
Before we can create vector index, we need to perform some admin tasks. There are two services we need to connect to from Oracle database: OpenAI and OCI Object Storage.
To create connection to OpenAI, we need to provide credentials. The value for the password parameter is Secret Key, that you generate for your profile setting on OpenAI.
Generate a new secret key and copy/paste it into your script as password.
The second credential that is required is to connect to the vector store on OCI Object Storage. To connect to OCI Object storage OCI user username and password is required:
Make sure that username includes also identity domain as prefix: identity_domain/username. Password is authentication token is generated in OCI Console (navigate to My Profile, click on Auth tokens, then Generate token and follow the steps and instructions). Copy/paste password into script.
Then we need an AI profile. This profile among others specifies the vector index we are about to create and to use for RAG.
We are now ready to create a new vector index. We will specify the location of our files that we are using for prompt augmentation. Please pay attention to the URL format. Because we are using credential with username and password, URL has to be in swift object storage format (hidden part of the URL refers to namespace)
Vector index is usually created very fast. Once create, we need to active the new profile:
Testing
We have seen that one of the text files that I've uploaded to OCI Object Storage is talking about kangaroos. So let's ask a question of how long a kangaroo lives?
We can see that response consists of two parts:
- actual generated answer: The average life expectancy of a kangaroo is about 4-6 years. However, in captivity, kangaroos may live as long as 20 years.
- reference to a file that actually contains required information: Sources: - files/S08_set1_a1.txt.txt (https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/frllu0v1kplh/select-ai-rag-data/files/S08_set1_a1.txt.txt)
Another example is for instance a question about Gustav Klimt, famous Austrian painter:
Now, the response is a bit more complex. There are actually more the one response, embedding model has identified 5 documents which have been used to augment the prompt (question tell me about gustav klimt).
In image below, the same response is in a form of the table. Beside generated response and source file that was used in prompt augmentation, score is presented - the higher score, the higher influence specific document has on the response.
Conclusion
In this very simple example, we can see that it is relatively easy to set up an object store and enable it for RAG by using vector index in Oracle 23ai database. With a few steps, you can enrich your PL/SQL database to become GenAI enabled. You just have to bring one of popular Large Language Models and register it with Oracle 23ai database. In one of my future blogs, I'll test it how all of these perform on non-english texts.
References
Complete Notebook Script
%script
BEGIN
DBMS_CLOUD.CREATE_CREDENTIAL(
credential_name => 'OPENAI_CRED35',
username => 'OPENAI',
password => '******************'
);
EXCEPTION
WHEN others THEN
DBMS_OUTPUT.PUL_LINE(sqlerrm);
END;
BEGIN
DBMS_CLOUD.DROP_CREDENTIAL('OCI_CRED');
DBMS_CLOUD.CREATE_CREDENTIAL(
credential_name => 'OCI_CRED',
username => '**********************',
password => '*************'
);
END;
BEGIN
DBMS_CLOUD_AI.DROP_PROFILE('OPENAI_ORACLE');
DBMS_CLOUD_AI.CREATE_PROFILE(
profile_name =>'OPENAI_ORACLE',
attributes =>'{"provider": "openai",
"credential_name": "OPENAI_CRED",
"vector_index_name": "RAG_DEMO_IDX",
"temperature": 0.2,
"max_tokens": 4096,
"model": "gpt-3.5-turbo-1106"
}');
END;
BEGIN
DBMS_CLOUD_AI.CREATE_VECTOR_INDEX(
index_name => 'RAG_DEMO_IDX',
attributes => '{"vector_db_provider": "oracle",
"location": "https://swiftobjectstorage.eu-frankfurt-1.oraclecloud.com/v1/<namespace>/select-ai-rag-data",
"object_storage_credential_name": "OCI_CRED",
"profile_name": "OPENAI_ORACLE",
"vector_dimension": 1536,
"vector_distance_metric": "cosine",
"chunk_overlap":128,
"chunk_size":1024
}');
END;
BEGIN
DBMS_CLOUD_AI.SET_PROFILE(
profile_name => 'OPENAI_ORACLE'
);
END;