In my blog post Oracle AI Vision: Just like a Box of Chocolates I am discussing how to create a custom machine learning model for pneumonia image classification (can an image provided be classified as normal or infected lungs) using Oracle’s OCI Vision AI service for code-less image classification machine learning model.
In this post, I am taking the model created during my previous exercise and plugging it into Oracle Analytics in order to make predictions from there using self-service Oracle Analytics tools such as Data Flows and Data Visualisation.
But before we can start using OCI Vision models in Oracle Analytics, there is a small amount of setup work to be done.
Note: Oracle documentation for Analytics Cloud contains very good guidance on how to set everything up. You can find this document here. My example below mostly follows that document, however use case is different and there are one or two things which the original document is not mentioning (safe domains, plugins).
Set Policies
To begin with, let me explain how my cloud services are organised by compartments.
My Analytics instance is located in a compartment called OracleAnalyticsBootcamp. Users who are using this instance are grouped in a group called OracleAnalyticsBootcampGroup.
OCI Vision AI service and Object Storage are both located in another compartment, called DataLakehouse. OracleAnalyticsBootcampGroup needs access to both services.
Access and privileges are managed by setting proper policies.
Access to OCI Vision Service
To use OCI Vision Services, the following policy needs to be set:
allow group OracleAnalyticsBootcampGroup to manage ai-service-vision-family in tenancy
This policy should be set on the root level compartment.
Staging Bucket
OCI Vision requires additional staging bucket that needs to be created. Simply navigate to Object Storage and create a new Bucket. There are no specific requirements for that.
Just, an additional comment: in this storage, the results of each prediction are temporarily stored, so eventually these can be accessed:
The JSON file for image IM-0115-0001.jpg can be view/downloaded and has the following content:
{"labels":[{"name":"NORMAL","confidence":0.8927698},{"name":"PNEUMONIA","confidence":0.10723018}],"ontologyClasses":[{"name":"PNEUMONIA","parentNames":[],"synonymNames":[]},{"name":"NORMAL","parentNames":[],"synonymNames":[]}],"imageClassificationModelVersion":"version","errors":[]}
This example is exactly the response from the model we would get if we ran prediction using OCI Vision web UI for that particular image; 89,27% confidence that lungs in image are normal, and 10,72% indicates prognosis of “Pneumonia".
Privileges for accessing buckets
To access, read and manage objects in a staging bucket and buckets where images are stored, the following policies are required:
for staging bucket:
allow group OracleAnalyticsBootcampGroup to read buckets in compartment DataLakehouse
allow group OracleAnalyticsBootcampGroup to manage objects in compartment DataLakehouse where any {request.permission='OBJECT_CREATE', request.permission='OBJECT_INSPECT'}
and for bucket with new images (in this case both, staging and new images bucket are in the same compartment, but this does not have to be the case):
allow group OracleAnalyticsBootcampGroup to read buckets in compartment DataLakehouse
Now, it is all set to connect Oracle Analytics to OCI Vision.
Connect Oracle Analytics to OCI Vision
To be able to analyse images using the model created in OCI Vision (for example Pneumonia model), a new connection to OCI Vision services has to be set.
Create connection has now a new option to connect to OCI Resource:
- Click Generate and then Copy
- Navigate to User Settings (right top icon in OCI console)
- Add API Key (just copied) in API Keys
And once done, save new Connection. There should be “success” message displayed.
Prepare images in Object storage
In most cases new images, eg. images that are required to be analysed, are stored somewhere. In OCI, this will most likely be Object Storage.
For example, we can set up a new bucket, eg. called Pneumonia_OAC. This is the place where all images are uploaded for further processing.
Before continuing, there is a small step required to make sure that images from Object Storage are accessible and presentable to Oracle Analytics.
The domain of your tenancy has to be whitelisted in the Safe Domains list which is defined Oracle Analytics (Safe Domains are under Console).
There should be an enrty *.eu-frankfurt-1.oraclecloud.com (in my case!) domain.
Checkboxes Image and Connect need to be checked in order to connect to image library in Object Storage and to display those images in Oracle Analytics.
Register OCI Vision model with Oracle Analytics
We are almost at the end. What needs to be done next is to register OCI Vision model, for example Pneumonia model, with Oracle Analytics.
Next to Create button is Page menu icon (three dots).
Selecting OCI Vision Models from Register Model/Functions opens the Register a Vision Model dialog. This is a 4-step dialog which will register an existing Vision model with Oracle Analytics.
First, select a Connection. In our case, this is OCI Vision connection.
Then select a model from the list of models. In our case, we will use Pneumonia model after Oracle patch.
Details of the selected model are displayed and can be reviewed. What is needed though is to provide the name of the staging bucket created earlier.
In our case, staging bucket is called AIVisionModelStagingBucket.
Click Register to complete.
Registered model is now listed in the Machine Learning Models list and is ready to be used:
Perform image classification using Data Flows
OCI Vision model can be applied in Data Flows. When using OCI Vision web UI, only one image can be uploaded and classified. In Oracle Analytics this is performed a bit differently.
The process starts with the list of images, direct links to images stored in object storage or link to the bucket that contains images.
The list of objects (bucket or images) is provided by storing them in a new dataset in Oracle Analytics. Dataset can be created simply by uploading one of the two CSV files, for example:
- buckets.csv:
1;Pneumonia;https://cloud.oracle.com/object-storage/buckets/...
- images.csv:
1;IM-1427-0001.jpeg;https://objectstorage.../val_NORMAL_NORMAL2-IM-1427-0001.jpeg
2;IM-1430-0001.jpeg;https://objectstorage.../val_NORMAL_NORMAL2-IM-1430-0001.jpeg
…
11;bacteria_4881.jpeg;https://objectstorage.../val_PNEUMONIA_person1950_bacteria_4881.jpeg
12;bacteria_4883.jpeg;https://objectstorage.../val_PNEUMONIA_person1952_bacteria_4883.jpeg
When one or the other dataset is ready, then a new data flow that would apply registered OCI Vision model is created.
In our example, I will show example with the bucket list, but data flow with image list as a data source is identical. So, let’s create a new Data Flow.
For Data is OCI Vison - Bucket dataset selected.
As a 2nd step Apply AI Model step is used.
Choosing Apply AI Model step opens a short dialog window in which AI Model is selected. In our case, we see only the model we just registered.
After the AI Model is added as a new step, there are a few parameters to be set. Eventually, we can leave them all as they are, except we need to select an Input column. This is the column containing the path to images. In our case, this column is Bucket URL from bucket.csv file.
The last step in a new data flow is to save the results into a new dataset, eg. OCI Vision - Classification.
We can already see the results at the bottom of the page. Of course, a new data flow has still to be saved and executed.
Vision Series Plug-in
This next step is not mandatory, if there is no plan to visualise images in Data Visualisation.
So, before we continue with data visualisation part of the project, we need a visualisation to display images. For this purpose, there is a plug-in available at Oracle Analytics Extensions Library.
Download and install it as a new Extension in your Oracle Analytics instance.
Visualise and Analyse
I guess, now we are ready to visualise and analyse predictions. For example:
The left part of the visualisation is a simple pivot table containing image name and location and confidence for Pneumonia or Normal lungs.
However for the right part Vision Series plug-in is used. Let’s take a look at it.
Visualisation's parameters for our example are rather simple. We are showing only one image at the time, and Image File from Bucket is located in Image Location field. If we were using this visualisation component for object detection, then object coordinates would be put into Vertices field.
As image is displayed one by one (always is displayed the 1st from the list), the pivot table visualisation is used as a filter.
Conclusion
Well, yet another "box of chocolates", with very tasty chocolates inside.
The process of integrating Oracle Analytics with OCI Vision models is smooth and I haven’t observed any issues. The whole process in fact from start to end (with the exception of images bulk labelling before training the model) is code-less and done in self-service mode. Once policies are set and connections are established, end users can perform more or less all the task by themselves. Which in my view is quite powerful.
Let’s assume OCI Vision model is replaced with some AutoML generated models for “regular” business data. We would need a bigger box for all those chocolates.