AWS Bedrock with OpenWebUI

AWS Bedrock lacks variety in web UI options. But there is a way to connect it to the OpenWebUI. In following article I'll show you a way to do it.

AWS Bedrock

AWS Bedrock is a managed service by Amazon Web Services that lets developers build, train, and deploy machine learning models. It offers pre-trained models and scalable infrastructure, simplifying the integration of AI into applications.

Problem

AWS Bedrock, as it’s awesome and pretty cheap way of accessing variety of LLMs without having to run server on your own, which can be pretty pricey, it lacks decent web UI. I was searching around to find something useful, but just couldn’t find it.

What I did find though is a project called: Bedrock Access Gateway, which is basically OpenAI-compatible RESTful API for Amazon Bedrock.

Putting it all together

Bedrock Access Gateway

Let’s get our Bedrock Access Gateway first to get it up and running, lets not use default Dockerfile, but Dockerfile_ecs:

git clone https://github.com/aws-samples/bedrock-access-gateway.git
cd bedrock-access-gateway/src/
rm Dockerfile
mv Dockerfile_ecs Dockerfile
docker build -t bedrock-proxy .

We will need AWS access so let’s enter proper access credentials by running:

aws configure

Now that we have our credentials set, let’s bring up our bedrock-proxy and pass it credentials to get access to AWS Bedrock:

docker run --rm -v $HOME/.aws/config:/root/.aws/config:ro -d -p 8081:80 bedrock-proxy       

OpenWebUI

Now lets bring OpenWebUI up and running by executing:

docker run --rm -d -p 8080:8080 -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:ollama

Configuring

Lets spin up our http://localhost:8080 in web browser, register and login to the OpenWebUI. Click on your profile icon on top right and choose: Admin Panel → Settings → Connections

Disable Ollama API and edit the OpenAI API host to match your docker local ip and path /api/v1 as seen on the screenshot. Default password for Bedrock Access Gateway is: bedrock

Here we go, lets test it, as you can see on top we see bunch of available models on AWS Bedrock:

I chose anthropic.claude-3-sonnet-20240229-v1:0, but you can choose any other.

Voila!

Troubleshooting

If you get an error like this, you probably didnt subscribe to the model in AWS model access.

To verify it, you can check logs of bedrock-proxy container and you will see:

    | fastapi.exceptions.HTTPException: 500: An error occurred (AccessDeniedException) when calling the ConverseStream operation: You don't have access to the model with the specified model ID.

To enable it, just go to web console and under AWS Bedrock service, subscribe and add the model you wish to use. More info here.