|
|
|
> 从mysql数据库中读取
|
|
|
|
|
|
|
|
**class**参数配置为```sql.SqlDocReader```
|
|
|
|
示例:
|
|
|
|
```yaml
|
|
|
|
sql_qichacha_push:
|
|
|
|
class: sql.SqlDocReader
|
|
|
|
init:
|
|
|
|
db:
|
|
|
|
host: 192.168.109.220
|
|
|
|
port: 3306
|
|
|
|
database: collie
|
|
|
|
user: collie
|
|
|
|
password: eill
|
|
|
|
|
|
|
|
query:
|
|
|
|
table:
|
|
|
|
a: company_lawsuit
|
|
|
|
b: company_lawsuit_parsed_info
|
|
|
|
columns:
|
|
|
|
a: "*"
|
|
|
|
b:
|
|
|
|
- id as bid
|
|
|
|
- jd_case_reason
|
|
|
|
- plaintiffs
|
|
|
|
- defendants
|
|
|
|
- thirdParties
|
|
|
|
condition: a.uuid = b.lawsuit_uuid
|
|
|
|
|
|
|
|
offset:
|
|
|
|
field: id
|
|
|
|
store: "file:///home/collie/project-collie/offset_company_lawsuit.txt"
|
|
|
|
```
|
|
|
|
* **db**: mysql数据库连接配置
|
|
|
|
* **query**: 查询条件,支持直接完整sql语句、指定表名和列名两种方式
|
|
|
|
1. 完整sql语句
|
|
|
|
* **query_sql**: select查询语句,其中where子句必须包含**$CONDITIONS**关键字
|
|
|
|
2. 指定表名和列名
|
|
|
|
* **query.table**: 数据表,支持单表,多表join查询
|
|
|
|
* **query.columns**: select查询的列名列表,逗号(,)分割。默认是所有列
|
|
|
|
* **query.condition**: select查询的where条件
|
|
|
|
* **offset**: 记录上次查询结束时记录在表中的offset
|
|
|
|
1. **offset.field**: 记录offset的字段名称
|
|
|
|
2. **offset.store**: offset保存的路径,支持本地文件和mysql数据库,前缀分别是```file://```、```mysql://``` |