Using Amazon S3 Vectors with Node.js

Amazon S3 Vectors provides vector storage and similarity search directly in Amazon S3. It can be useful when we need vector search for AI applications without managing a separate vector database.

In this example, I am using Node.js and the AWS SDK.

Install AWS SDK

Install the S3 Vectors client package:

npm install @aws-sdk/client-s3vectors

Create the client

Import and create the S3 Vectors client for your AWS Region.

import {
    S3VectorsClient
} from "@aws-sdk/client-s3vectors";

const client = new S3VectorsClient({
    region: "ap-south-1"
});

The SDK will use your normal AWS credential chain. For local development, this can be credentials configured using the AWS CLI.

Create a vector bucket

First create a vector bucket.

import {
    CreateVectorBucketCommand
} from "@aws-sdk/client-s3vectors";

await client.send(
    new CreateVectorBucketCommand({
        vectorBucketName: "my-vector-bucket"
    })
);

A vector bucket is different from a normal S3 general purpose bucket. It is designed specifically for storing and querying vectors.

Create an index

Inside the vector bucket, create an index and define the vector dimensions.

import {
    CreateIndexCommand
} from "@aws-sdk/client-s3vectors";

await client.send(
    new CreateIndexCommand({
        vectorBucketName: "my-vector-bucket",
        indexName: "documents",
        dimension: 1024,
        distanceMetric: "cosine",
        dataType: "float32"
    })
);

The dimension must match the embedding model being used.

For example, if our embedding model returns 1024 values, the S3 Vectors index must also use 1024 dimensions.

Add vectors

Once we have embeddings, they can be stored using PutVectorsCommand.

import {
    PutVectorsCommand
} from "@aws-sdk/client-s3vectors";

await client.send(
    new PutVectorsCommand({
        vectorBucketName: "my-vector-bucket",
        indexName: "documents",
        vectors: [
            {
                key: "doc-1",
                data: {
                    float32: embedding
                },
                metadata: {
                    title: "My first document"
                }
            }
        ]
    })
);

Here embedding is an array of numbers generated by an embedding model.

The metadata can store useful information about the vector, such as document name, user ID, category or tags.

Search vectors

To find similar vectors, use QueryVectorsCommand.

import {
    QueryVectorsCommand
} from "@aws-sdk/client-s3vectors";

const result = await client.send(
    new QueryVectorsCommand({
        vectorBucketName: "my-vector-bucket",
        indexName: "documents",
        queryVector: {
            float32: queryEmbedding
        },
        topK: 5,
        returnMetadata: true
    })
);

console.log(result.vectors);

queryEmbedding should be generated using the same embedding model used while storing the vectors.

S3 Vectors will compare it with the stored vectors and return the closest matches.

Simple AI memory flow

One practical use of S3 Vectors is long-term memory for an AI application.

The flow can be kept simple:

User text
   ↓
Embedding model
   ↓
S3 Vectors
   ↓
Similarity search
   ↓
Relevant memories
   ↓
LLM

For an AWS application, the embedding can come from Amazon Bedrock and the resulting vector can be stored in S3 Vectors.

This gives us a simple serverless approach without running a dedicated vector database.

IAM permissions

The application also needs permission to work with the vector bucket and index.

Keep the IAM policy limited to the operations actually required by the application, such as creating indexes, adding vectors and querying vectors.

For production applications, avoid giving the application broad S3 permissions just because the service has S3 in its name.

When S3 Vectors makes sense

S3 Vectors is interesting when we need:

  • semantic search
  • AI agent memory
  • RAG document retrieval
  • recommendation or similarity search
  • large vector collections where keeping infrastructure simple matters

For applications needing very advanced vector database features or extremely low-latency search, a dedicated vector database may still be a better fit.

But for many AWS-native AI applications, S3 Vectors gives us another useful option with very little infrastructure to manage.

Total
0
Shares
Leave a Reply

Your email address will not be published. Required fields are marked *