semitechnologies/weaviate

The AI-native database for a new generation of software

https://pkg.go.dev/github.com/weaviate/weaviate https://github.com/weaviate/weaviate/actions/workflows/.github/workflows/pull_requests.yaml https://goreportcard.com/report/github.com/weaviate/weaviate https://codecov.io/gh/weaviate/weaviate

https://github.com/weaviate-tutorials/

Weaviate is an open-source vector database that simplifies the development of AI applications. Built-in vector and hybrid search, easy-to-connect machine learning models, and a focus on data privacy enable developers of all levels to build, iterate, and scale AI capabilities faster.

To get started quickly, have a look at one of these pages:

• Quickstart tutorial To see Weaviate in action
• Contributor guide To contribute to this project

For more details, read through the summary on this page or see the system documentation.

Why Weaviate?

Weaviate uses state-of-the-art machine learning (ML) models to turn your data - text, images, and more - into a searchable vector database.

Here are some highlights.

Speed

Weaviate is fast. The core engine can run a 10-NN nearest neighbor search on millions of objects in milliseconds. See benchmarks.

Flexibility

Weaviate can vectorize your data at import time. Or, if you have already vectorized your data, you can upload your own vectors instead.

Modules give you the flexibility to tune Weaviate for your needs. More than two dozen modules connect you to popular services and model hubs such as OpenAI, Cohere, VoyageAI and HuggingFace. Use custom modules to work with your own models or third party services.

Production-readiness

Weaviate is built with scaling, replication, and security in mind so you can go smoothly from rapid prototyping to production at scale.

Beyond search

Weaviate doesn't just power lightning-fast vector searches. Other superpowers include recommendation, summarization, and integration with neural search frameworks.

Who uses Weaviate?

• Software Engineers

  • Weaviate is an ML-first database engine
  • Out-of-the-box modules for AI-powered searches, automatic classification, and LLM integration
  • Full CRUD support
  • Cloud-native, distributed system that runs well on Kubernetes
  • Scales with your workloads

• Data Engineers

  • Weaviate is a fast, flexible vector database
  • Use your own ML model or third party models
  • Run locally or with an inference service

• Data Scientists

  • Seamless handover of Machine Learning models to engineers and MLOps
  • Deploy and maintain your ML models in production reliably and efficiently
  • Easily package custom trained models

What can you build with Weaviate?

A Weaviate vector database can search text, images, or a combination of both. Fast vector search provides a foundation for chatbots, recommendation systems, summarizers, and classification systems.

Here are some examples that show how Weaviate integrates with other AI and ML tools:

Use Weaviate with third party embeddings

• Cohere (blogpost)
• Hugging Face
• https://github.com/openai/openai-cookbook/tree/main/examples/vector_databases/weaviate

Use Weaviate as a document store

• DocArray
• https://docs.haystack.deepset.ai/reference/integrations-weaviate#weaviatedocumentstore (https://www.deepset.ai/weaviate-vector-search-engine-integration)

Use Weaviate as a memory backend

• https://github.com/Significant-Gravitas/Auto-GPT/blob/master/docs/configuration/memory.md#weaviate-setup (blogpost)
• LangChain (blogpost)
• https://gpt-index.readthedocs.io/en/latest/how_to/integrations/vector_stores.html (blogpost)
• https://github.com/openai/***-retrieval-plugin/blob/main/docs/providers/weaviate/setup.md

Demos

These demos are working applications that highlight some of Weaviate's capabilities. Their source code is available on GitHub.

• Verba, the Golden RAGtreiver (https://github.com/weaviate/verba)
• Healthsearch (https://github.com/weaviate/healthsearch-demo)
• Awesome-Moviate (https://github.com/weaviate-tutorials/awesome-moviate)

How can you connect to Weaviate?

Weaviate exposes a GraphQL API and a REST API. Starting in v1.23, a new gRPC API provides even faster access to your data.

Weaviate provides client libraries for several popular languages:

• Python
• JavaScript/TypeScript
• Go
• Java

There are also community supported libraries for additional languages.

Where can You learn more?

Free, self-paced courses in Weaviate Academy teach you how to use Weaviate. The https://github.com/weaviate-tutorials has code for example projects. The https://github.com/weaviate/recipes has even more project code to get you started.

The Weaviate blog and podcast regularly post stories on Weaviate and AI.

Here are some popular posts:

Blogs

• What to expect from Weaviate in 2023
• Why is vector search so fast?
• Cohere Multilingual ML Models with Weaviate
• Vamana vs. HNSW - Exploring ANN algorithms Part 1
• HNSW+PQ - Exploring ANN algorithms Part 2.1
• The Tile Encoder - Exploring ANN algorithms Part 2.2
• How GPT4.0 and other Large Language Models Work
• Monitoring Weaviate in Production
• The *** Retrieval Plugin - Weaviate as a Long-term Memory Store for Generative AI
• Combining LangChain and Weaviate
• How to build an Image Search Application with Weaviate
• Cohere Multilingual ML Models with Weaviate
• Building Multimodal AI in TypeScript
• Giving Auto-GPT Long-Term Memory with Weaviate

Podcasts

• Neural Magic in Weaviate
• BERTopic
• Jina AI's Neural Search Framework

Other reading

• Weaviate is an open-source search engine powered by ML, vectors, graphs, and GraphQL (ZDNet)
• Weaviate, an ANN Database with CRUD support (DB-Engines.com)
• A sub-50ms neural search with DistilBERT and Weaviate (Towards Datascience)
• Getting Started with Weaviate Python Library (Towards Datascience)

Join our community!

At Weaviate, we love to connect with our community. We love helping amazing people build cool things. And, we love to talk with you about you passion for vector databases and AI.

Please reach out, and join our community:

• Community forum
• https://github.com/weaviate/weaviate
• Slack
• X (***)

To keep up to date with new releases, meetup news, and more, subscribe to our newsletter