Enhancing Privacy in Vector Search with Cyborg and RAPIDS

In today’s rapidly evolving landscape of generative AI, vector databases have emerged as crucial tools for efficiently storing and querying high-dimensional data. However, just like any database, these vector databases are exposed to various threats, including cyber attacks, phishing scams, and unauthorized access. This risk is especially alarming given that many of these databases are laden with sensitive and confidential information. 🔒

Addressing Security Concerns in Vector Databases

To tackle the pressing issue of security, Cyborg has partnered with NVIDIA to bolster the protection of vector databases using the RAPIDS cuVS library. This open-source toolkit enhances vector search with cutting-edge algorithms, bringing GPU acceleration to Cyborg’s encrypted vector search engine. The objective of this collaboration is to ensure robust security without sacrificing performance.

The Vulnerabilities of Vector Databases

Vector databases are integral to modern, data-intensive applications, supporting functions from retrieval-augmented generation (RAG) pipelines to recommendation systems. Their high-performance indexing and search functionalities make them invaluable, but the sensitive nature of the data they house attracts malicious parties. This risk of exposure is particularly critical for sectors that demand confidentiality:

• Regulated Industries: Sectors such as healthcare, financial services, and the public sector face stringent privacy and security mandates that may restrict the use of vector search and its associated applications.
• Intellectual Property-Driven Sectors: Industries like pharmaceuticals, manufacturing, and defense rely heavily on intellectual property as key components of their competitive edge and operational value.

While security concerns can be overlooked during the prototyping phase of AI-driven workloads, they can become major obstacles when moving into production.

Introducing Confidential Vector Search

Cyborg, a New York-based startup, has created an end-to-end encrypted vector search engine to tackle these issues. By leveraging forward privacy and cryptographic hashing, Cyborg Vector Search facilitates the secure indexing and retrieval of confidential data. This end-to-end encryption ensures that no unencrypted vectors are ever stored in the database, thereby significantly minimizing the attack surface and addressing the aforementioned confidentiality concerns.

The design of Cyborg Vector Search balances several key performance metrics:

• End-to-End Encryption: This guarantees the utmost level of security and confidentiality through a cryptographically secure architecture tailored for stringent privacy requirements.
• High Performance: It minimizes the incremental cost of encryption, achieving cryptographic overhead of less than 5% for indexing and under 30% for retrieval.
• Compatibility: The solution maintains compatibility with existing vector search pipelines, allowing for an easy transition from prototype to production.

NVIDIA Hardware Integration

To enable encrypted indexing on GPUs, NVIDIA Confidential Computing technology is employed. This framework ensures that data remains secure through cryptography and robust access controls, leveraging trusted execution environments (TEEs) to create a secure space for sensitive operations. Such technology is essential for protecting data confidentiality during GPU-accelerated computations.

The core hardware behind this innovative solution is the NVIDIA H100 Tensor Core GPU (80 GB) with Confidential Computing capabilities. This technology is available on all NVIDIA Hopper Tensor Core GPUs and will continue into the next generation with NVIDIA Blackwell Tensor Core GPUs.

The NVIDIA GPUs utilized in Confidential Computing mode come equipped with hardware-based cryptographic engines, firewalls, and remote attestation flows activated to guarantee the integrity of the TEE. This ensures that end users can confirm and validate that their confidential workloads are secured while in operation on the GPU.

NVIDIA Hopper Confidential Computing employs AES-GCM256 encryption and signature capabilities for user data on the PCIe bus, while also blocking infrastructure and out-of-band access with firewalls configured through signed and attestable firmware. NVIDIA goes further by providing a public remote attestation service, which allows end users to receive ongoing assurances that their drivers and firmware have not been compromised due to vulnerabilities.

Cyborg quickly developed their design using NVIDIA LaunchPad, a platform that provides users with hands-on access to prebuilt labs in a browser-based sandbox. The design was preconfigured with all the necessary guidelines and steps to ensure that the system was built and configured appropriately for confidential workloads.

Accelerating Confidential Vector Search

Much like conventional vector searching, confidential vector searching is resource-intensive and can be challenging to scale. This makes it an ideal candidate for GPU acceleration. The RAPIDS cuVS library includes highly optimized algorithms specifically designed to enhance this process.

To assess the performance improvements enabled by this integration, Cyborg and NVIDIA conducted a joint proof-of-concept study that involved the incorporation of cuVS with Cyborg Vector Search. This effort aimed to realize GPU-accelerated encrypted vector searching.

The proof-of-concept compared the performance of encrypted indexing and retrieval on CPUs versus GPUs. Specifically, the analysis substituted scikit-learn KMeans and hashlib on a CPU with cuVS and a custom SHA-3 CUDA kernel on a GPU, yielding remarkable results:

• The average index build time improved by an impressive 47x, reducing the time required to index vector embeddings from several hours to mere minutes. The steps accelerated by cuVS achieved an even more dramatic enhancement of 52.2x for clustering model training and inference.
• Retrieval also experienced significant enhancements: the cuVS-accelerated sections of the pipeline delivered a 9.8x performance improvement with minimal code modifications.
• When enabling NVIDIA Hopper Confidential Computing modes for complete encryption during indexing and retrieval, the overhead costs were only marginal—1-2% for indexing and 15-25% for retrieval, far outweighed by the gains from GPU acceleration.

Performance Metrics with GPU Acceleration

The index build process is critical, particularly the phase of clustering model training, which often dominates the overall timeline. However, even by excluding training and concentrating solely on quantization and encrypted indexing, the GPU still demonstrated significant performance advantages.

The comprehensive retrieval pipeline also saw major time reductions with the GPU in play. All calculations were performed on the same index configuration, ensuring a recall level greater than 0.95. The IVFPQ index type was chosen for its exceptional blend of efficiency and accuracy.

In an age where data breaches can have severe consequences, the security of vector databases is a fundamental requirement for many organizations. The partnership between Cyborg’s Vector Search, RAPIDS cuVS, and NVIDIA Confidential Computing presents a robust solution to augment the security of vector databases while maintaining high performance levels. As the landscape continues to evolve, solutions like these play a pivotal role in ensuring that sensitive data remains protected and secure. 🛡️