... | ... | @@ -212,23 +212,26 @@ scrapy |
|
|
|
|
|
## 责任人
|
|
|
|
|
|
```html
|
|
|
```
|
|
|
范召贤
|
|
|
```
|
|
|
|
|
|
## 数据归集方式
|
|
|
|
|
|
- [ ] 爬虫直接写kafka
|
|
|
|
|
|
- [ ] 爬虫写文件logstash采集
|
|
|
- [x] 爬虫写文件logstash采集
|
|
|
|
|
|
## 爬虫结果目录
|
|
|
|
|
|
```html
|
|
|
```
|
|
|
/data/gravel_spiders/certifications
|
|
|
```
|
|
|
|
|
|
## 归集后存放目录
|
|
|
|
|
|
```html
|
|
|
```
|
|
|
/data2_227/grvael_spider_result/certifications
|
|
|
```
|
|
|
|
|
|
## logstash配置文件名称
|
... | ... | @@ -244,11 +247,13 @@ scrapy |
|
|
## 数据归集的topic
|
|
|
|
|
|
```
|
|
|
general-taxpayer
|
|
|
```
|
|
|
|
|
|
## ES日志索引及筛选条件
|
|
|
|
|
|
```
|
|
|
gravel-spider-data-*
|
|
|
```
|
|
|
|
|
|
## 监控指标看板
|
... | ... | |