eosphorosai/dbgpt
eosphorosai
DBGPT Docker镜像仓库是一个集中存储和分发DBGPT相关Docker镜像的资源库，旨在为用户提供便捷、高效的部署体验。该仓库包含多种类型的镜像，如基础运行环境镜像、完整应用镜像及特定功能模块镜像，支持不同版本的DBGPT快速部署，适用于开发、测试及生产等多种场景。通过使用仓库中的镜像，用户可大幅简化环境配置流程，减少部署复杂度，同时确保获取到经过验证的稳定版本，助力DBGPT相关应用的快速落地与高效运维。
15 次收藏下载次数: 0状态：社区镜像维护者：eosphorosai仓库类型：镜像最近更新：3 个月前
让 AI 帮你使用轩辕镜像？ · 展开查看说明 · 点击收起说明
如果你使用 DeepSeek、元宝 AI、Copilot、ChatGPT 等 AI 工具，协助配置轩辕镜像、编写 docker pull 命令、修改 Docker Compose 镜像地址、配置镜像加速、排查镜像拉取失败、分析报错日志等问题，请先让 AI 阅读并遵守轩辕镜像的规则文档。
只需在 AI 对话中先发送下面这句话即可：
请先完整阅读并严格遵守以下文档中的全部规则与要求：
https://xuanyuan.cloud/agents.md
在未充分阅读并理解该文档前，不要生成任何命令、配置、修改建议、故障排查方案或技术回答。后续所有输出都必须严格以该文档中的规范为最高优先级执行。查看 agents.md 用法指南与完整示范。国内用户首推 元宝 AI、DeepSeek 的深度思考模式，不推荐豆包 AI；Cursor 等编辑器可在对话 @ 该链接，或加入 User Rules。 若 AI 无法访问外链，可 打开说明文档 复制全文粘贴。文档会随站点更新，复制内容可能过期，建议定期检查。
DB-GPT: Revolutionizing Database Interactions with Private LLM Technology
https://github.com/eosphoros-ai/DB-GPT
https://github.com/eosphoros-ai/DB-GPThttps://github.com/eosphoros-ai/DB-GPThttps://github.com/eosphoros-ai/DB-GPThttps://db-gpt.readthedocs.io/en/latest/%7Chttps://github.com/eosphoros-ai/DB-GPT/blob/main/README.zh.md#%E8%81%94%E7%B3%BB%E6%88%91%E4%BB%AC%7Chttps://github.com/eosphoros-ai/community
What is DB-GPT?
DB-GPT is an experimental open-source project that uses localized GPT large models to interact with your data and environment. With this solution, you can be assured that there is no risk of data leakage, and your data is 100% private and secure.
Contents
- Install
- Demo
- introduction
- features
- contribution
- roadmap
- contact
DB-GPT *** Video
Demo
Run on an RTX 4090 GPU.
Chat Excel
!https://github.com/eosphoros-ai/DB-GPT/assets/***/0474d220-2a9f-449f-a940-92c8a25af390
Chat Plugin
!https://github.com/eosphoros-ai/DB-GPT/assets/***/7d95c347-f4b7-4fb6-8dd2-c1c02babaa56
LLM Management
!https://github.com/eosphoros-ai/DB-GPT/assets/***/501d6b3f-c4ce-4197-9a6f-f016f8150a11
FastChat && vLLM
!https://github.com/eosphoros-ai/DB-GPT/assets/***/0c9475d2-45ee-4573-aa5a-814f7fd40213
Trace
!https://github.com/eosphoros-ai/DB-GPT/assets/***/69bd14b8-14d0-4ca9-9cb7-6cef44a2bc93
Chat Knowledge
!https://github.com/eosphoros-ai/DB-GPT/assets/***/72266a48-edef-4c6d-88c6-fbb1a24a6c3e
Install
!https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge&logo=docker&logoColor=white !https://img.shields.io/badge/Linux-FCC624?style=for-the-badge&logo=linux&logoColor=black !https://img.shields.io/badge/mac%20os-000000?style=for-the-badge&logo=macos&logoColor=F0F0F0 !https://img.shields.io/badge/Windows-0078D6?style=for-the-badge&logo=windows&logoColor=white
https://db-gpt.readthedocs.io/en/latest/getting_started/install/deploy/deploy.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/deploy.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/deploy.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/docker/docker.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/docker_compose/docker_compose.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/chatdb/chatdb.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/chatdb/chatdb.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/kbqa/kbqa.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/chatexcel/chatexcel.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/dashboard/dashboard.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/model/model.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/application/chatagent/chatagent.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/cluster/cluster.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/cluster/vms/standalone.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/cluster/vms/index.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/llm/vllm/vllm.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/observability.html
- https://db-gpt.readthedocs.io/en/latest/getting_started/faq/deploy/deploy_faq.html
Features
Currently, we have released multiple key features, which are listed below to demonstrate our current capabilities:
-
Private KBQA & data processing
The DB-GPT project offers a range of features to enhance knowledge base construction and enable efficient storage and retrieval of both structured and unstructured data. These include built-in support for uploading multiple file formats, the ability to integrate plug-ins for custom data extraction, and unified vector storage and retrieval capabilities for managing large volumes of information.
-
Multiple data sources & visualization
The DB-GPT project enables seamless natural language interaction with various data sources, including Excel, databases, and data warehouses. It facilitates effortless querying and retrieval of information from these sources, allowing users to engage in intuitive conversations and obtain insights. Additionally, DB-GPT supports the generation of analysis reports, providing users with valuable summaries and interpretations of the data.
-
Multi-Agents&Plugins
It supports custom plug-ins to perform tasks, natively supports the Auto-GPT plug-in model, and the Agents protocol adopts the Agent Protocol standard.
-
Fine-tuning text2SQL
An automated fine-tuning lightweight framework built around large language models, Text2SQL data sets, LoRA/QLoRA/Pturning, and other fine-tuning methods, making TextSQL fine-tuning as convenient as an assembly line. https://github.com/eosphoros-ai/DB-GPT-Hub
-
Multi LLMs Support, Supports multiple large language models, currently supporting
Massive model support, including dozens of large language models such as open source and API agents. Such as LLaMA/LLaMA2, Baichuan, ChatGLM, Wenxin, Tongyi, Zhipu, etc.
- https://huggingface.co/Tribbiani/vicuna-13b
- https://huggingface.co/lmsys/vicuna-13b-v1.5
- https://huggingface.co/meta-llama/Llama-2-7b-chat-hf
- https://huggingface.co/baichuan-inc
- https://huggingface.co/baichuan-inc/baichuan-7B
- https://huggingface.co/THUDM/chatglm-6b
- https://huggingface.co/THUDM/chatglm2-6b
- https://huggingface.co/tiiuae/falcon-40b
- https://huggingface.co/internlm/internlm-chat-7b
- https://huggingface.co/Qwen/
- https://huggingface.co/BlinkDL/rwkv-4-raven
- https://huggingface.co/camel-ai/CAMEL-13B-Combined-Data
- https://huggingface.co/databricks/dolly-v2-12b
- https://huggingface.co/h2oai/h2ogpt-gm-oasst1-en-2048-open-llama-7b
- https://huggingface.co/lmsys/fastchat-t5
- https://huggingface.co/mosaicml/mpt-7b-chat
- https://huggingface.co/nomic-ai/gpt4all-13b-snoozy
- https://huggingface.co/NousResearch/Nous-Hermes-13b
- https://huggingface.co/Salesforce/codet5p-6b
- https://huggingface.co/timdettmers/guanaco-33b-merged
- https://huggingface.co/WizardLM/WizardLM-13B-V1.0
- https://huggingface.co/WizardLM/WizardCoder-15B-V1.0
- https://huggingface.co/FlagAlpha/Llama2-Chinese-13b-Chat
- https://huggingface.co/VMware/open-llama-7b-open-instruct
Etc.
- Support API Proxy LLMs
- https://api.openai.com/
- https://www.aliyun.com/product/dashscope
- https://cloud.baidu.com/product/wenxinworkshop?track=dingbutonglan
- http://open.bigmodel.cn/
-
Privacy and security
The privacy and security of data are ensured through various technologies, such as privatized large models and proxy desensitization.
-
Support Datasources
| DataSource | support | Notes |
|---|---|---|
| https://www.mysql.com/ | Yes | |
| https://www.postgresql.org/ | Yes | |
| https://github.com/apache/spark | Yes | |
| https://github.com/duckdb/duckdb | Yes | |
| https://github.com/sqlite/sqlite | Yes | |
| https://github.com/microsoft/mssql-jdbc | Yes | |
| https://github.com/ClickHouse/ClickHouse | Yes | |
| https://github.com/oracle | No | TODO |
| https://github.com/redis/redis | No | TODO |
| https://github.com/mongodb/mongo | No | TODO |
| https://github.com/apache/hbase | No | TODO |
| https://github.com/apache/doris | No | TODO |
| https://github.com/IBM/Db2 | No | TODO |
| https://github.com/couchbase | No | TODO |
| https://github.com/elastic/elasticsearch | No | TODO |
| https://github.com/OceanBase | No | TODO |
| https://github.com/pingcap/tidb | No | TODO |
| https://github.com/StarRocks/starrocks | No | TODO |
Introduction
The architecture of the entire DB-GPT is shown.
The core capabilities mainly consist of the following parts:
- Multi-Models: Support multi-LLMs, such as LLaMA/LLaMA2、CodeLLaMA、ChatGLM, QWen、Vicuna and proxy model ***、Baichuan、tongyi、wenxin etc
- Knowledge-Based QA: You can perform high-quality intelligent Q&A based on local documents such as PDF, word, excel, and other data.
- Embedding: Unified data vector storage and indexing, Embed data as vectors and store them in vector databases, providing content similarity search.
- Multi-Datasources: Used to connect different modules and data sources to achieve data flow and interaction.
- Multi-Agents: Provides Agent and plugin mechanisms, allowing users to customize and enhance the system's behavior.
- Privacy & Secure: You can be assured that there is no risk of data leakage, and your data is 100% private and secure.
- Text2SQL: We enhance the Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models
RAG-IN-Action
SubModule
- https://github.com/eosphoros-ai/DB-GPT-Hub Text-to-SQL performance by applying Supervised Fine-Tuning (SFT) on large language models.
- https://github.com/eosphoros-ai/DB-GPT-Plugins DB-GPT Plugins Can run autogpt plugin directly
- https://github.com/eosphoros-ai/DB-GPT-Web ChatUI for DB-GPT
Image
🌐 AutoDL Image
Language Switching
In the .env configuration file, modify the LANGUAGE parameter to switch to different languages. The default is English (Chinese: zh, English: en, other languages to be added later).
Contribution
- Please run
black .before submitting the code. Contributing guidelines, https://github.com/csunny/DB-GPT/blob/main/CONTRIBUTING.md
RoadMap
!https://github.com/eosphoros-ai/DB-GPT/blob/main/assets/roadmap.jpg
KBQA RAG optimization
-
Multi Documents
- Excel, CSV
- Word
- Text
- MarkDown
- Code
- Images
-
RAG
-
Graph Database
- Neo4j Graph
- Nebula Graph
-
Multi-Vector Database
- Chroma
- Milvus
- Weaviate
- PGVector
- Elasticsearch
- ClickHouse
- Faiss
-
Testing and Evaluation Capability Building
- Knowledge QA datasets
- Question collection [easy, medium, hard]:
- Scoring mechanism
- Testing and evaluation using Excel + DB datasets
Multi Datasource Support
- Multi Datasource Support
- MySQL
- PostgreSQL
- Spark
- DuckDB
- Sqlite
- MSSQL
- ClickHouse
- Oracle
- Redis
- MongoDB
- HBase
- Doris
- DB2
- Couchbase
- Elasticsearch
- OceanBase
- TiDB
- StarRocks
Multi-Models And vLLM
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/cluster/vms/index.html
- https://github.com/lm-sys/FastChat
- https://db-gpt.readthedocs.io/en/latest/getting_started/install/llm/vllm/vllm.html
- Cloud-native environment and support for Ray environment
- Service Registry(eg:nacos)
- Compatibility with OpenAI's interfaces
- Expansion and optimization of embedding models
Agents market and Plugins
- multi-agents framework
- custom plugin development
- plugin market
- Integration with CoT
- Enrich plugin sample library
- Support for AutoGPT protocol
- Integration of multi-agents and visualization capabilities, defining LLM+Vis new standards
Cost and Observability
- https://db-gpt.readthedocs.io/en/latest/getting_started/observability.html
- Observability
- cost & budgets
Text2SQL Finetune
-
support llms
- LLaMA
- LLaMA-2
- BLOOM
- BLOOMZ
- Falcon
- Baichuan
- Baichuan2
- InternLM
- Qwen
- XVERSE
- ChatGLM2
-
SFT Accuracy
As of October 10, 2023, by fine-tuning an open-source model of 13 billion parameters using this project, the execution accuracy on the Spider evaluation dataset has surpassed that of GPT-4!
| name | Execution Accuracy | reference |
|---|---|---|
| GPT-4 | 0.762 | numbersstation-eval-res |
| *** | 0.728 | numbersstation-eval-res |
| CodeLlama-13b-Instruct-hf_lora | 0.789 | sft train by our this project,only used spider train dataset ,the same eval way in this project with lora SFT |
| CodeLlama-13b-Instruct-hf_qlora | 0.774 | sft train by our this project,only used spider train dataset ,the same eval way in this project with qlora and nf4,bit4 SFT |
| wizardcoder | 0.610 | https://github.com/cuplv/text-to-sql-wizardcoder/tree/main |
| CodeLlama-13b-Instruct-hf | 0.556 | eval in this project default param |
| llama2_13b_hf_lora_best | 0.744 | sft train by our this project,only used spider train dataset ,the same eval way in this project |
https://github.com/eosphoros-ai/DB-GPT-Hub
Licence
The MIT License (MIT)
oldzhang
运维工程师
Linux服务器
5
"Docker访问体验非常流畅，大镜像也能快速完成下载。"