Skip to content

GitLab

pdd_find_goods · Changes

Page history
update: 拼多多搜索商品数据入库和表结构 authored by 李子健's avatar 李子健
Hide whitespace changes
Inline Side-by-side
Showing with 183 additions and 0 deletions
lake/pdd_find_goods.md 0 → 100644
View page @ c3e92cde
### 数据存储
```
阿里云Mysql
```
- host: bdp-ec.rwlb.rds.aliyuncs.com
- user: ***
- password: ***
- database: bdp_pdd
- table: pdd_find_goods
### 建表语句
```sql
CREATE TABLE if not exists bdp_pdd.`pdd_find_goods` (
`id` int(11) NOT NULL AUTO_INCREMENT COMMENT '自增主键',
`platform_name` varchar(16) DEFAULT NULL COMMENT '平台名称',
`goods_id` varchar(64) DEFAULT NULL COMMENT '商品ID',
`mall_id` varchar(64) default null comment '店铺ID',
`goods_name` varchar(64) default null comment '商品名称',
`short_name` varchar(64) default null comment '商品名称简称',
`sales` int(11) default null comment '销量',
`group_sales` varchar(16) default null comment '拼团销量',
`group_price` int(11) default null comment '商品拼团价，单位：分',
`price_type` smallint default null comment '价格类型',
`normal_price` int(11) default null comment '商品原价，单位：分',
`market_price` int(11) default null comment '商品市场价，单位：分',
`merchant_type` smallint default null comment '店主类型',
`brand_id` varchar(64) default null comment '品牌id',
`is_brand` smallint(6) default '0' comment '品牌标签，0非品牌，1品牌',
`tag_list` text default null comment '商品特征描述',
`search_key` varchar(16) default null comment '搜索关键词',
`goods_url` varchar(255) default null comment '商品链接',
`mall_url` varchar(255) default null comment '店铺链接',
`thumb_url` varchar(255) default null,
`hd_url` varchar(255) default null,
`hd_thumb_url` varchar(255) default null,
`tag_style` varchar(8) default null,
`event_type` varchar(8) default null,
`activity_type` varchar(8) default null,
`tag` varchar(8) default null,
`hd_thumb_wm` varchar(64) default null,
`is_app` varchar(8) default null,
`list_type` varchar(8) default null,
`image_wm` varchar(64) default null,
`thumb_wm` varchar(64) default null,
`create_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '收录时间',
`lastupdatetime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT '最后更新时间',
PRIMARY KEY (`id`) ,
UNIQUE KEY `IX_UNIQUE_NAME_ID` (`goods_id`),
KEY `IX_SEARCH_KEY` (`search_key`),
KEY `IX_SHORT_NAME` (`short_name`),
KEY `IX_SALES` (`sales`),
KEY `IX_SALES_TIP` (`group_sales`),
KEY `IX_PRICE` (`group_price`),
KEY `IX_MALL_ID` (`mall_id`),
KEY `IX_PLATFORM_SHOP_ID` (`goods_name`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8 COMMENT='拼多多关键字搜索商品数据';
nohup python /home/collie/product/app_conv_ebusiness/jobs/data_pump/data_pump.py pump-data --config /home/collie/product/app_conv_ebusiness/data_pump/conv_pdd_kw_goods.yml &
```
### 数据结构
```json
{
"id": "自增主键",
"platform_name": "平台名称",
"goods_id": "商品ID",
"mall_id": "店铺ID",
"goods_name": "商品名称",
"short_name": "商品名称简称",
"sales": "销量",
"group_sales": "拼团销量",
"group_price": "商品拼团价，单位：分",
"price_type": "价格类型",
"normal_price": "商品原价，单位：分",
"market_price": "商品市场价，单位：分",
"merchant_type": "店主类型",
"brand_id": "品牌id",
"is_brand": "品牌标签，0非品牌，1品牌",
"tag_list": "商品特征描述",
"search_key": "搜索关键词",
"goods_url": "商品链接",
"mall_url": "店铺链接",
"thumb_url": "",
"hd_url": "",
"hd_thumb_url": "",
"tag_style": "",
"event_type": "",
"activity_type": "",
"tag": "",
"hd_thumb_wm": "",
"is_app": "",
"list_type": "",
"image_wm": "",
"thumb_wm": "",
"create_time": "收录时间",
"lastupdatetime": "最后更新时间"
}
```
### 数据字典
| 字段名 | 字段类型 | 注释 |
| -------------- | --------- | ---------------------------------------------------- |
| id | int | 自增主键 |
| platform_name | varchar | 平台名称 |
| goods_id | varchar | 商品ID |
| mall_id | varchar | 店铺ID |
| goods_name | varchar | 商品名称 |
| short_name | varchar | 商品名称简称 |
| sales | int | 销量 |
| group_sales | varchar | 拼团销量 |
| group_price | int | 商品拼团价，单位：分 |
| price_type | smallint | 价格类型 |
| normal_price | int | 商品原价，单位：分 |
| market_price | int | 商品市场价，单位：分 |
| merchant_type | smallint | 店主类型 |
| brand_id | varchar | 品牌id |
| is_brand | smallint | 品牌标签，0非品牌，1品牌 |
| tag_list | text | 商品特征描述 |
| search_key | varchar | 搜索关键词 |
| goods_url | varchar | 商品链接 |
| mall_url | varchar | 店铺链接 |
| icon_url | varchar | 图标链接 |
| thumb_url | varchar | APP下载链接 |
| hd_url | varchar | APP官方主页 |
| hd_thumb_url | varchar | APP状态，0： 初始（未采集详情） 1：正常 9：APP已下线 |
| tag_style | varchar | APP标签（多个标签用“,”隔开） |
| event_type | varchar | 收录时间 |
| activity_type | varchar | 最后更新时间 |
| tag | varchar | 使用标记，10废弃使用 |
| hd_thumb_wm | varchar | 是否是历史数据，1历史数据 |
| is_app | varchar | |
| list_type | varchar | |
| image_wm | varchar | |
| thumb_wm | varchar | |
| create_time | timestamp | 收录时间 |
| lastupdatetime | timestamp | 最后更新时间 |
### 样例
```json
{
"id": 382584,
"platform_name": "pinduoduo",
"goods_id": "268811522037",
"mall_id": "305217126",
"goods_name": "GIORGIOARMANI阿玛尼轻垫精华粉底液大红气垫234精华粉底液bb霜",
"short_name": "GIORGIOARMANI阿玛尼轻垫精华粉底液大红气垫234精华粉底液bb霜",
"sales": 1,
"group_sales": "已拼1件",
"group_price": 31000,
"price_type": 0,
"normal_price": 39000,
"market_price": 58000,
"merchant_type": 5,
"brand_id": "1028",
"is_brand": 0,
"tag_list": "[text: 退货包运费,text_color: #FF5706,type: 3,tag_track_info:1&3]",
"search_key": "阿玛尼粉底气垫",
"goods_url": "https://mobile.yangkeduo.com/goods.html?goods_id=268811522037",
"mall_url": "http://mobile.pinduoduo.com/mall_page.html?mall_id=305217126",
"thumb_url": "https://img.pddpic.com/gaudit-image/2021-08-10/88091cac182fe457d78fd843b29a5d5a.jpeg",
"hd_url": "https://img.pddpic.com/mms-material-img/2021-08-10/cdb49ebf-040f-4f60-bc27-b961037dd673.jpeg.a.jpeg",
"hd_thumb_url": "https://img.pddpic.com/gaudit-image/2021-08-10/747e67888c2de887f1efde921505d4ac.jpeg",
"tag_style": "0",
"event_type": "0",
"activity_type": "0",
"tag": "-1",
"hd_thumb_wm": "",
"is_app": "0",
"list_type": "0",
"image_wm": "",
"thumb_wm": "",
"create_time": "2021-08-27 15:42:33",
"lastupdatetime": "2021-08-27 15:42:33"
}
```
### 说明
\ No newline at end of file