Building a RAG application with AWS Bedrock

Building a RAG application with AWS Bedrock

Brain John Aboze

February 05, 2025

This blog post was written by Brain John Aboze as part of the Deepchecks Community Blog. If you would like to contribute your own blog post, feel free to reach out to us via blog@deepchecks.com. We typically pay a symbolic fee for content that's accepted by our reviewers.

Introduction

Retrieval-augmented generation (RAG) represents an advanced AI framework that improves language models by using retrieval systems to generate relevant and accurate responses, whether domain-specific or general. RAG systems help bridge the gap between information retrieval and generative tasks and, at the same time, offer applications in sectors such as healthcare, finance, and customer service.

The demand for precise AI applications is seen in the growth of the generative AI market. With an estimated rising compound annual growth rate (CAGR) of 35%, a new McKinsey analysis projects the worldwide generative AI market to reach $110 billion by 2030. A study published in Nature Machine Intelligence shows that RAG applications can raise response accuracy by 20–30% compared with single generative models. This development immediately affects sectors that depend on real-time data for decision-making processes, including e-commerce and logistics, where effective AI systems can cut running costs by up to 25%.

AWS Bedrock is seen as a modern platform that simplifies the implementation of RAG workflows. The infrastructure of AWS Bedrock reduces the time-to-market for AI applications and offers scalability, simplifying the deployment of these solutions. Most LLMs, such as GPT or Llama models, are trained on data with a cut-off date, typically 2022 or 2023. Their lack of knowledge of current events makes them unfit for real-time or domain-specific work. RAG addresses this by including a retrieval system that offers accurate and current LLM data. RAG improves LLMs by allowing them to query knowledge bases or outside databases for pertinent data, therefore producing results with increased accuracy and context relevance. This improved accuracy and efficiency highlight a need for AI systems that are capable of real-time decision-making, which traditional language models struggle to achieve.

With AWS Bedrock, developers can implement scalable and efficient RAG processes without the complexity of managing infrastructure.

Implementing RAG using Amazon Bedrock Knowledge Base.

Building Process

Step 1: Prepare your data

The first step toward a good RAG solution is data preparation. Documents, PDFs, proprietary reports, or other text material must be kept in an Amazon S3 bucket, which is a scalable cloud storage service provided by AWS for storing and retrieving large amounts of data. Supported file formats include .txt, .md, .html, .docx, .csv, .xls, and .pdf.

For example, a financial company might post quarterly financial reports or market analyses to its S3 bucket. Healthcare firms can similarly save patient instructions or medical procedures. These records are the major data source for retrieval from AWS Bedrock, so the knowledge base ensures pertinent and domain-specific material.

Step 2: Select the LLM model

The available options for Meta’s Llama 3.1 models on AWS Bedrock include the Llama 3.1 405B Instruct model. It supports multilingual capabilities and extended 128K context lengths. Before setting up your Knowledge Base in AWS Bedrock, you need to request access to an LLM. Here, the Llama 3.1 405B Instruct model is used, as it is an open-source model, and it can be used for different applications.

Testing Amazon Bedrock Knowledge Base with Llama 3.1 405B

Step 3: Set up a Knowledge Base

AWS Bedrock streamlines the building of a knowledge base, the repository for embedding and retrieval. The Titan Text Embeddings v2 model, a high-performance embedding model offered by AWS, is commonly selected for its balance of accuracy and cost-effectiveness. The knowledge base setup includes:

After having access to the foundation model, such as Meta’s Llama 3.1 405B Instruct, go to the Amazon Bedrock console and select the option to create a knowledge base. During this step, you will finalize the integration by selecting an embedding model, linking your data source, and configuring vector storage for retrieval. Once the knowledge base is developed, you can test it by querying text chunks to verify foundation model responses. Finally, you can easily include the knowledge base in your application or apply it to AI agents to enable RAG and other sophisticated processes.

Using AWS console to create Knowledge Bases on Amazon Bedrock

For instance, a company managing energy reports can upload documents to its knowledge base and use OpenSearch to find consumer insights.

Step 4: Integrate retrieval and generation

The RetrieveAndGenerate API is the core of the RAG workflow in AWS Bedrock. This API first retrieves relevant data from the knowledge base and then feeds it into the foundation model to generate accurate, context-aware responses. Here is the workflow:

For example, a stock analyst querying “What are Gulf Energy’s latest financial updates?” would receive a synthesized, accurate answer based on the documents stored in the knowledge base.

AWS Bedrock Knowledge Base interface showcasing the integration of the RAG with Llama 3.1 405B Instruct model

Step 5: Test the workflow

After selecting the desired model and provisioning your knowledge base, the next step is to test it. You can test the knowledge base directly through the Amazon Bedrock console or via an API call, using the selected model to ensure that it retrieves and processes data accurately. This testing phase is used to verify that the knowledge base integrates well with the foundation model and produces contextually relevant outputs for your queries.

Testing ensures the solution is functional and aligned with business needs. While AWS provides tools for managing RAG workflows, external tools like Streamlit, an open-source framework for building web applications, can be used to create custom interfaces for interacting with the RAG system. Streamlit is not part of AWS but can be easily integrated into the workflow to provide a user-friendly front end. For instance, you can design a simple UI with buttons to trigger queries, view responses, and display retrieved documents. For example, a financial analyst could enter a query in Streamlit, and the RAG system would retrieve data from the knowledge base to generate a detailed report instantly. This integration allows real-time decision-making with accurate, updated data, supporting AWS Bedrock’s backend capabilities while offering a streamlined user experience.

Now, let’s introduce some real examples of applications.

Applications

Simplifying the infrastructure and integration process, AWS Bedrock enables developers to produce solutions that are scalable, context-aware, and efficient. One of the applications that should be highlighted is in the domain of financial analysis with RAG. AWS Bedrock provides financial firms with the ability to create RAG systems that effortlessly combine retrieval and generation. One example on GitHub shows how to design a question-answering application that searches relevant financial information from an Amazon S3-hosted knowledge base. With the Llama 3.1 model available on AWS Bedrock, the application responds in real time to consumer questions concerning financial data, investment strategies, or market developments. This connection guarantees correct decision-making and helps to save time and running expenses. Two data sources are used for this project: one for content published on a website and another for documents kept on Amazon S3. Amazon OpenSearch Serverless generates vector storage in a vector search collection.

The following figure shows the architecture of a chatbot solution that uses AWS Bedrock. It presents how components, including Amazon S3, AWS Lambda, Amazon OpenSearch Serverless, and the knowledge base for Amazon Bedrock, work together to store and retrieve data for generating context-aware responses. This figure highlights how data flows from the input (uploaded documents or user queries) through the system in order to enable lean interaction between retrieval and generation processes.

Solution architecture for chatbot

Solution architecture for web datasource

This figure presents the architecture for a web data source integration using AWS Bedrock. It shows how web content is managed through AWS services, including AWS Lambda and API Gateway, intending to retrieve, process, and ingest web data into the knowledge base for Amazon Bedrock. This architecture is used for applications that require continuous updates from web sources in order to guarantee that the knowledge base remains up-to-date with relevant and current information.

Another example includes applications in E-Commerce. AWS Bedrock enables e-commerce applications with real-time information and recommendations. This application shows how to develop a vector database for storing and retrieving embeddings by using Amazon Bedrock with OpenSearch. This architecture enables companies to rapidly get contextually relevant data, hence improving their recommendation engines and customer service solutions.

Using AWS Bedrock with the LangChain architecture shows how to create a RAG application for PDF document querying. In this case, Titan Text Embeddings from AWS Bedrock index and embed PDFs are kept in Amazon S3. This case study applies to sectors like legal processes and academic research, where users can enter natural language searches, and the system provides summaries and finds pertinent materials.

Extending RAG with tools in AWS Bedrock

With simplified deployment and operational procedures, AWS Bedrock provides tools and techniques to enable meeting specific needs. Amazon Bedrock’s knowledge bases offer a completely automated RAG solution with data intake and retrieval. This simplifies the RAG pipeline by removing the need for custom code to link data sources with LLMs. For instance, the ai-chat-accelerator project, a chatbot solution, combines the Bedrock knowledge base to provide semantic search by ingesting unstructured documents kept on Amazon S3. This project emphasizes how to document the chunking and parsing techniques of AWS Bedrock to help RAG processes be more effective.

As the ai-chat-accelerator project shows, integrating tools and infrastructure is another approach to extending the RAG workflow. This project uses:

  1. Store and manage vector embeddings and user interaction data using PostSQL (via Amazon Aurora Serverless), hence allowing persistence and query efficiency.
  2. AWS Fargate containers to run modular Python-based apps, therefore streamlining orchestration and improving scalability.
  3. Terraform among infrastructure-as-code (IaC) solutions that help to effectively implement the whole RAG application stack.

The orchestrator logic can be customized to include extra capabilities, including semantic hybrid search, multi-turn conversational history, reranking, and filtering of search results. For corporate use cases such as customer service, internal knowledge management, and decision-making systems, these characteristics make RAG processes quite flexible.

The physical architecture

GraphRAG, which models complicated relationships between data points to improve precision and contextual comprehension, is among the most advanced options for expanding RAG processes. GraphRAG improves answer accuracy by up to 35% when using the graph’s capacity to represent semantic relationships instead of vector-only retrieval methods. GraphRAG can be implemented in AWS Bedrock by integrating graph databases alongside the knowledge base to store and query complex data relationships. For sectors like healthcare, where contextual relationships between medical terms and procedures are important, this architecture is specifically applicable.

Implementing GraphRAG

Conclusion

RAG provides unmatched efficiency, accuracy, and adaptability, whether it is improving diagnosis accuracy in healthcare, giving real-time investment insights in finance, or allowing dynamic customer service. The managed architecture of AWS Bedrock streamlines the implementation of these solutions, therefore enabling RAG to be available to companies and developers without demanding sophisticated technical knowledge. Bedrock’s infrastructure automates processes such as document ingestion, embedding management, and vector storage, in order to enable faster deployment of RAG applications.

Now is the time to act. Discover how RAG with AWS Bedrock might transform your processes regardless of your position – developer, business executive, or healthcare expert. Start by working with AWS Bedrock’s knowledge base, request access to foundation models like Llama 3.1, and use its strong APIs to apply custom RAG solutions. Accept this innovative technique if you want to keep ahead in a fast-changing, demanding market. Start today with AWS Bedrock!