lintoai/linto-platform-nlp-extractive-summarization

linto-platform-nlp-extractive-summarization

镜像概述和主要用途

本镜像用于构建LinTO的NLP服务：提取式摘要，基于https://github.com/linto-ai/linto-platform-nlp-core开发。可与https://github.com/linto-ai/linto-platform-stack一同部署，也可独立部署（详见下文开发部分）。

LinTO的NLP服务采用spaCy的核心设计理念：组件和管道。组件（位于components/目录下）与服务解耦，可轻松复用至其他spaCy项目，通过组织成管道实现特定NLP任务。

本服务支持两种启动方式：REST API和Celery任务，且提供GPU支持选项。

核心功能和特性

• 基于spaCy架构：采用组件和管道设计，组件可复用
• 双服务模式：支持REST API（HTTP）和Celery任务（task）两种运行模式
• 硬件适配：支持GPU加速和纯CPU运行
• 多语言支持：默认支持英语（en）和法语（fr）
• 可配置摘要参数：支持调整摘要比例、句子数量、聚类算法等参数

使用场景和适用范围

适用于需要从文本中提取关键信息生成摘要的场景，如：

• 新闻文章自动摘要
• 文档内容提炼
• 对话记录关键信息提取
• 可集成到LinTO平台生态或作为独立微服务部署

使用方法和配置说明

部署准备

1. 下载模型

在主机上将模型下载到./assets目录（也可存储在其他位置），确保已安装https://git-lfs.github.com/并可在`/usr/local/bin/git-lfs`访问。

bash
cd linto-platform-nlp-extractive-summarization/
bash scripts/download_models.sh

2. 配置环境变量

bash
cp .envdefault .env

构建镜像

bash
sudo docker build --tag lintoai/linto-platform-nlp-extractive-summarization:latest .

或使用docker-compose:

bash
sudo docker-compose build

运行容器

GPU支持模式

确保已安装https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html#installing-on-ubuntu-and-debian和GPU驱动。

bash
sudo docker run --gpus all \
--rm -p 80:80 \
-v $PWD/assets:/app/assets:ro \
--env-file .env \
lintoai/linto-platform-nlp-extractive-summarization:latest

纯CPU运行设置

• 从命令中移除--gpus all
• 在.env中设置USE_GPU=False

或使用docker-compose:

bash
sudo docker-compose up

纯CPU运行设置

• 从docker-compose.yml中移除runtime: nvidia
• 在.env中设置USE_GPU=False

服务验证

• 若SERVICE_MODE=http，可通过浏览器访问http://localhost/docs或http://localhost/redoc查看交互式API文档
• 若SERVICE_MODE=task，请参考文档末尾的Celery任务部分

API规范：http://localhost/extsumm/{lang}

支持的语言

请求格式

使用" | "（左右各有一个空格）分隔文本片段（如句子、段落、文档等），作为提取式摘要的基本单元。

json
{
  "articles": [
    {
      "text": "The Chrysler Building, the famous art deco New York skyscraper, will be sold for a small fraction of its previous sales price. | The deal, first reported by The Real Deal, was for $150 million, according to a source familiar with the deal. | Mubadala, an Abu Dhabi investment fund, purchased 90% of the building for $800 million in 2008. | Real estate firm Tishman Speyer had owned the other 10%. | The buyer is RFR Holding, a New York real estate company. | Officials with Tishman and RFR did not immediately respond to a request for comments. | It's unclear when the deal will close. | The building sold fairly quickly after being publicly placed on the market only two months ago. | The sale was handled by CBRE Group. | The incentive to sell the building at such a huge loss was due to the soaring rent the owners pay to Cooper Union, a New York college, for the land under the building. | The rent is rising from $7.75 million last year to $32.5 million this year to $41 million in 2028. | Meantime, rents in the building itself are not rising nearly that fast. | While the building is an iconic landmark in the New York skyline, it is competing against newer office towers with large floor-to-ceiling windows and all the modern amenities. | Still the building is among the best known in the city, even to people who have never been to New York. | It is famous for its triangle-shaped, vaulted windows worked into the stylized crown, along with its distinctive eagle gargoyles near the top. | It has been featured prominently in many films, including Men in Black 3, Spider-Man, Armageddon, Two Weeks Notice and Independence Day. | The previous sale took place just before the 2008 financial meltdown led to a plunge in real estate prices. | Still there have been a number of high profile skyscrapers purchased for top dollar in recent years, including the Waldorf Astoria hotel, which Chinese firm Anbang Insurance purchased in 2016 for nearly $2 billion, and the Willis Tower in Chicago, which was formerly known as Sears Tower, once the world's tallest. | Blackstone Group (BX) bought it for $1.3 billion 2015. | The Chrysler Building was the headquarters of the American automaker until 1953, but it was named for and owned by Chrysler chief Walter Chrysler, not the company itself. | Walter Chrysler had set out to build the tallest building in the world, a competition at that time with another Manhattan skyscraper under construction at 40 Wall Street at the south end of Manhattan. He kept secret the plans for the spire that would grace the top of the building, building it inside the structure and out of view of the public until 40 Wall Street was complete. | Once the competitor could rise no higher, the spire of the Chrysler building was raised into view, giving it the title."
    },
    {
      "text": "Junk foods taste good that’s why it is mostly liked by everyone of any age group especially kids and school going children. | They generally ask for the junk food daily because they have been trend so by their parents from the childhood. | They never have been discussed by their parents about the harmful effects of junk foods over health. | According to the research by scientists, it has been found that junk foods have negative effects on the health in many ways. | They are generally fried food found in the market in the packets. | They become high in calories, high in cholesterol, low in healthy nutrients, high in sodium mineral, high in sugar, starch, unhealthy fat, lack of protein and lack of dietary fibers. | Processed and junk foods are the means of rapid and unhealthy weight gain and negatively impact the whole body throughout the life. | It makes able a person to gain excessive weight which is called as obesity. | Junk foods tastes good and looks good however do not fulfil the healthy calorie requirement of the body. | Some of the foods like french fries, fried foods, pizza, burgers, candy, soft drinks, baked goods, ice cream, cookies, etc are the example of high-sugar and high-fat containing foods. | It is found according to the Centres for Disease Control and Prevention that Kids and children eating junk food are more prone to the type-2 diabetes. | In type-2 diabetes our body become unable to regulate blood sugar level. | Risk of getting this disease is increasing as one become more obese or overweight. | It increases the risk of kidney failure. | Eating junk food daily lead us to the nutritional deficiencies in the body because it is lack of essential nutrients, vitamins, iron, minerals and dietary fibers. | It increases risk of cardiovascular diseases because it is rich in saturated fat, sodium and bad cholesterol. | High sodium and bad cholesterol diet increases blood pressure and overloads the heart functioning. | One who like junk food develop more risk to put on extra weight and become fatter and unhealthier. | Junk foods contain high level carbohydrate which spike blood sugar level and make person more lethargic, sleepy and less active and alert. | Reflexes and senses of the people eating this food become dull day by day thus they live more sedentary life. | Junk foods are the source of constipation and other disease like diabetes, heart ailments, clogged arteries, heart attack, strokes, etc because of being poor in nutrition. | Junk food is the easiest way to gain unhealthy weight. | The amount of fats and sugar in the food makes you gain weight rapidly. | However, this is not a healthy weight. | It is more of fats and cholesterol which will have a harmful impact on your health. | Junk food is also one of the main reasons for the increase in obesity nowadays.This food only looks and tastes good, other than that, it has no positive points. | The amount of calorie your body requires to stay fit is not fulfilled by this food. | For instance, foods like French fries, burgers, candy, and cookies, all have high amounts of sugar and fats. | Therefore, this can result in long-term illnesses like diabetes and high blood pressure. | This may also result in kidney failure. | Above all, you can get various nutritional deficiencies when you don’t consume the essential nutrients, vitamins, minerals and more. | You become prone to cardiovascular diseases due to the consumption of bad cholesterol and fat plus sodium. | In other words, all this interferes with the functioning of your heart. | Furthermore, junk food contains a higher level of carbohydrates. | It will instantly spike your blood sugar levels. | This will result in lethargy, inactiveness, and sleepiness. | A person reflex becomes dull overtime and they lead an inactive life. | To make things worse, junk food also clogs your arteries and increases the risk of a heart attack. | Therefore, it must be avoided at the first instance to save your life from becoming ruined.The main problem with junk food is that people don’t realize its ill effects now. | When the time comes, it is too late. | Most importantly, the issue is that it does not impact you instantly. | It works on your overtime; you will face the consequences sooner or later. | Thus, it is better to stop now.You can avoid junk food by encouraging your children from an early age to eat green vegetables. | Their taste buds must be developed as such that they find healthy food tasty. | Moreover, try to mix things up. | Do not serve the same green vegetable daily in the same style. | Incorporate different types of healthy food in their diet following different recipes. | This will help them to try foods at home rather than being attracted to junk food.In short, do not deprive them completely of it as that will not help. | Children will find one way or the other to have it. | Make sure you give them junk food in limited quantities and at healthy periods of time."
    }
  ]
}

响应格式

json
{
  "extsumm": [
    {
      "text": "The Chrysler Building, the famous art deco New York skyscraper, will be sold for a small fraction of its previous sales price. | The deal, first reported by The Real Deal, was for $150 million, according to a source familiar with the deal. | Mubadala, an Abu Dhabi investment fund, purchased 90% of the building for $800 million in 2008. | Real estate firm Tishman Speyer had owned the other 10%. | The buyer is RFR Holding, a New York real estate company. | Officials with Tishman and RFR did not immediately respond to a request for comments. | It's unclear when the deal will close. | The building sold fairly quickly after being publicly placed on the market only two months ago. | The sale was handled by CBRE Group. | The incentive to sell the building at such a huge loss was due to the soaring rent the owners pay to Cooper Union, a New York college, for the land under the building. | The rent is rising from $7.75 million last year to $32.5 million this year to $41 million in 2028. | Meantime, rents in the building itself are not rising nearly that fast. | While the building is an iconic landmark in the New York skyline, it is competing against newer office towers with large floor-to-ceiling windows and all the modern amenities. | Still the building is among the best known in the city, even to people who have never been to New York. | It is famous for its triangle-shaped, vaulted windows worked into the stylized crown, along with its distinctive eagle gargoyles near the top. | It has been featured prominently in many films, including Men in Black 3, Spider-Man, Armageddon, Two Weeks Notice and Independence Day. | The previous sale took place just before the 2008 financial meltdown led to a plunge in real estate prices. | Still there have been a number of high profile skyscrapers purchased for top dollar in recent years, including the Waldorf Astoria hotel, which Chinese firm Anbang Insurance purchased in 2016 for nearly $2 billion, and the Willis Tower in Chicago, which was formerly known as Sears Tower, once the world's tallest. | Blackstone Group (BX) bought it for $1.3 billion 2015. | The Chrysler Building was the headquarters of the American automaker until 1953, but it was named for and owned by Chrysler chief Walter Chrysler, not the company itself. | Walter Chrysler had set out to build the tallest building in the world, a competition at that time with another Manhattan skyscraper under construction at 40 Wall Street at the south end of Manhattan. He kept secret the plans for the spire that would grace the top of the building, building it inside the structure and out of view of the public until 40 Wall Street was complete. | Once the competitor could rise no higher, the spire of the Chrysler building was raised into view, giving it the title.",
      "extractive_summary": [
        "The sale was handled by CBRE Group.",
        "Meantime, rents in the building itself are not rising nearly that fast.",
        "It is famous for its triangle-shaped, vaulted windows worked into the stylized crown, along with its distinctive eagle gargoyles near the top.",
        "Walter Chrysler had set out to build the tallest building in the world, a competition at that time with another Manhattan skyscraper under construction at 40 Wall Street at the south end of Manhattan. He kept secret the plans for the spire that would grace the top of the building, building it inside the structure and out of view of the public until 40 Wall Street was complete."
      ]
    },
    {
      "text": "Junk foods taste good that’s why it is mostly liked by everyone of any age group especially kids and school going children. | They generally ask for the junk food daily because they have been trend so by their parents from the childhood. | They never have been discussed by their parents about the harmful effects of junk foods over health. | According to the research by scientists, it has been found that junk foods have negative effects on the health in many ways. | They are generally fried food found in the market in the packets. | They become high in calories, high in cholesterol, low in healthy nutrients, high in sodium mineral, high in sugar, starch, unhealthy fat, lack of protein and lack of dietary fibers. | Processed and junk foods are the means of rapid and unhealthy weight gain and negatively impact the whole body throughout the life. | It makes able a person to gain excessive weight which is called as obesity. |