Python爬虫实例之淘宝商品页面爬取(api接口)

可以使用Python中的requests和BeautifulSoup库来进行网页爬取和数据提取。以下是一个简单的示例:
import requests
from bs4 import BeautifulSoup

def get_product_data(url):
    # 发送GET请求,获取网页内容
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0;Win64) AppleWebkit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3',
    }
    response = requests.get(url, headers=headers)
    if response.status_code == 200:
        # 使用BeautifulSoup解析网页内容
        soup = BeautifulSoup(response.text, 'html.parser')
        
        # 获取商品标题
        title = soup.select('.tb-main-title')[0].get_text().strip()
        
        # 获取商品价格
        price = soup.select('.tm-price')[0].get_text().strip()
        
        # 获取商品销量
        sales = soup.select('.tm-ind-panel .tm-count')[0].get_text().strip()
        
        # 返回提取的商品数据
        return {
            'title': title,
            'price': price,
            'sales': sales
        }
    else:
        print("请求失败!")

if __name__ == '__main__':
    # 测试爬取商品数据
    product_url = 'https://item.taobao.com/item.htm?spm=a230r.1.14.9.183b70b2UDwRMA&id=636877355813'
    data = get_product_data(product_url)
    print(data)

在代码中,我们首先定义了一个get_product_data()函数,该函数接受一个淘宝商品的URL作为参数。然后,我们发送GET请求获取网页内容,并使用BeautifulSoup库解析网页。

我们通过CSS选择器定位DOM元素,提取商品的标题、价格和销量等数据,并将其存储在一个字典中。最后,我们打印出提取的商品数据。

请注意,为了模拟正常浏览器的请求,我们在请求头中添加了User-Agent。这是为了防止被网站屏蔽或拒绝访问。

在实际应用中,你可以根据需要进行数据的进一步处理和存储,例如存储到数据库或导出为CSV文件。

以下是api封装好的展示示例

Python爬虫实例之淘宝商品页面爬取(api接口)_第1张图片

 返回数据:

{
	"item": {
		"num_iid": "705459759835",
		"title": "SYT气质女神 自制款短袖连衣裙女夏季新款高品质紧身显瘦包臀长裙",
		"desc_short": "",
		"price": 158,
		"total_price": "",
		"suggestive_price": "",
		"orginal_price": 158,
		"nick": "1112tx",
		"num": 200,
		"detail_url": "https://item.taobao.com/item.htm?id=705459759835",
		"pic_url": "https://img.alicdn.com/bao/uploaded/i2/870481044/O1CN01gnu7gX1JaCJNOFZBu_!!870481044.jpg",
		"brand": null,
		"brandId": "",
		"rootCatId": "",
		"cid": 50010850,
		"desc": "
\n \n
\n \n \n \n \n \n \n
\n
\n
\n \"好价节\" \n \n
\n
\n
\n \n
\n
\n \n
\n \n \n \n \n \n \n
\n
\n
\n \"八宫格关联\" \n \n
\n
\n
\n \n
\n

\n
\n 1910990958\n
", "item_imgs": [ { "url": "https://img.alicdn.com/bao/uploaded/i2/870481044/O1CN01gnu7gX1JaCJNOFZBu_!!870481044.jpg" }, { "url": "https://img.alicdn.com/imgextra/i4/870481044/O1CN01kc82x31JaCJJupFIy_!!870481044.jpg" }, { "url": "https://img.alicdn.com/imgextra/i2/870481044/O1CN01bx1B4u1JaCJLp0kh8_!!870481044.jpg" }, { "url": "https://img.alicdn.com/imgextra/i2/870481044/O1CN01t1uYxH1JaCJCYfNBn_!!870481044.jpg" }, { "url": "https://img.alicdn.com/imgextra/i2/870481044/O1CN01W3PfUh1JaCJFhNE43_!!870481044.jpg" } ], "item_weight": "", "post_fee": "5.00", "freight": "", "express_fee": "", "ems_fee": "", "shipping_to": "", "video": { "url": "http://cloud.video.taobao.com/play/u/p/1/e/6/t/1/399400641694.mp4" }, "sample_id": "", "props_name": "20509:28314:尺码:S;20509:28314:尺码:S;20509:28314:尺码:S;20509:28314:尺码:S;20509:28314:尺码:S;20509:28314:尺码:S;20509:28314:尺码:S;20509:28315:尺码:M;20509:28315:尺码:M;20509:28315:尺码:M;20509:28315:尺码:M;20509:28315:尺码:M;20509:28315:尺码:M;20509:28315:尺码:M;20509:28316:尺码:L;20509:28316:尺码:L;20509:28316:尺码:L;20509:28316:尺码:L;20509:28316:尺码:L;20509:28316:尺码:L;20509:28316:尺码:L;20509:28317:尺码:XL;20509:28317:尺码:XL;20509:28317:尺码:XL;20509:28317:尺码:XL;20509:28317:尺码:XL;20509:28317:尺码:XL;20509:28317:尺码:XL;1627207:28320:颜色分类:白色;1627207:28327:颜色分类:酒红色;1627207:3232479:颜色分类:深紫色;1627207:28341:颜色分类:黑色;1627207:28335:颜色分类:绿色;1627207:30155:颜色分类:杏色;1627207:30226:颜色分类:浅粉色;1627207:28320:颜色分类:白色;1627207:28327:颜色分类:酒红色;1627207:3232479:颜色分类:深紫色;1627207:28341:颜色分类:黑色;1627207:28335:颜色分类:绿色;1627207:30155:颜色分类:杏色;1627207:30226:颜色分类:浅粉色;1627207:28320:颜色分类:白色;1627207:28327:颜色分类:酒红色;1627207:3232479:颜色分类:深紫色;1627207:28341:颜色分类:黑色;1627207:28335:颜色分类:绿色;1627207:30155:颜色分类:杏色;1627207:30226:颜色分类:浅粉色;1627207:28320:颜色分类:白色;1627207:28327:颜色分类:酒红色;1627207:3232479:颜色分类:深紫色;1627207:28341:颜色分类:黑色;1627207:28335:颜色分类:绿色;1627207:30155:颜色分类:杏色;1627207:30226:颜色分类:浅粉色", "prop_imgs": { "prop_img": [ { "properties": "1627207:28320", "url": "//gd3.alicdn.com/imgextra/i4/870481044/O1CN01kc82x31JaCJJupFIy_!!870481044.jpg" }, { "properties": "1627207:28327", "url": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01t1uYxH1JaCJCYfNBn_!!870481044.jpg" }, { "properties": "1627207:3232479", "url": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01Hy2bEg1JaCJ3euABW_!!870481044.jpg" }, { "properties": "1627207:28341", "url": "//gd3.alicdn.com/imgextra/i2/870481044/O1CN01W3PfUh1JaCJFhNE43_!!870481044.jpg" }, { "properties": "1627207:28335", "url": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01r6sBzb1JaCJMajq8y_!!870481044.jpg" }, { "properties": "1627207:30155", "url": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01MgARg51JaCJKma7hO_!!870481044.jpg" }, { "properties": "1627207:30226", "url": "//gd2.alicdn.com/imgextra/i1/870481044/O1CN01nreksE1JaCJmGgaY4_!!870481044.jpg" } ] }, "props_imgs": { "prop_img": [ { "properties": "1627207:28320", "url": "//gd3.alicdn.com/imgextra/i4/870481044/O1CN01kc82x31JaCJJupFIy_!!870481044.jpg" }, { "properties": "1627207:28327", "url": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01t1uYxH1JaCJCYfNBn_!!870481044.jpg" }, { "properties": "1627207:3232479", "url": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01Hy2bEg1JaCJ3euABW_!!870481044.jpg" }, { "properties": "1627207:28341", "url": "//gd3.alicdn.com/imgextra/i2/870481044/O1CN01W3PfUh1JaCJFhNE43_!!870481044.jpg" }, { "properties": "1627207:28335", "url": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01r6sBzb1JaCJMajq8y_!!870481044.jpg" }, { "properties": "1627207:30155", "url": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01MgARg51JaCJKma7hO_!!870481044.jpg" }, { "properties": "1627207:30226", "url": "//gd2.alicdn.com/imgextra/i1/870481044/O1CN01nreksE1JaCJmGgaY4_!!870481044.jpg" } ] }, "property_alias": "", "props": [ { "name": "品牌", "value": "other/其他" }, { "name": "适用年龄", "value": "25-29周岁" }, { "name": "尺码", "value": "S M L XL" }, { "name": "图案", "value": "纯色" }, { "name": "风格", "value": "通勤" }, { "name": "通勤", "value": "复古" }, { "name": "领型", "value": "方领" }, { "name": "腰型", "value": "高腰" }, { "name": "衣门襟", "value": "拉链" }, { "name": "颜色分类", "value": "白色 酒红色 深紫色 黑色 绿色 杏色 浅粉色" }, { "name": "袖型", "value": "常规" }, { "name": "组合形式", "value": "单件" }, { "name": "裙型", "value": "包臀裙" }, { "name": "适用季节", "value": "夏季" }, { "name": "年份季节", "value": "2023年夏季" }, { "name": "袖长", "value": "短袖" }, { "name": "裙长", "value": "长裙" }, { "name": "流行元素/工艺", "value": "抽褶 褶皱 波浪 拉链" }, { "name": "廓形", "value": "A型" }, { "name": "材质成分", "value": "其他材质" } ], "total_sold": "1000", "skus": { "sku": [ { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:28320", "properties_name": "20509:28314:尺码:S;1627207:28320:颜色分类:白色", "quantity": 200, "sku_id": "4958597356171" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:28327", "properties_name": "20509:28314:尺码:S;1627207:28327:颜色分类:酒红色", "quantity": 200, "sku_id": "4958597356174" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:3232479", "properties_name": "20509:28314:尺码:S;1627207:3232479:颜色分类:深紫色", "quantity": 200, "sku_id": "4958597356177" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:28341", "properties_name": "20509:28314:尺码:S;1627207:28341:颜色分类:黑色", "quantity": 200, "sku_id": "4958597356180" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:28335", "properties_name": "20509:28314:尺码:S;1627207:28335:颜色分类:绿色", "quantity": 200, "sku_id": "5136958387148" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:30155", "properties_name": "20509:28314:尺码:S;1627207:30155:颜色分类:杏色", "quantity": 200, "sku_id": "4964918757567" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28314;1627207:30226", "properties_name": "20509:28314:尺码:S;1627207:30226:颜色分类:浅粉色", "quantity": 200, "sku_id": "5150786423327" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:28320", "properties_name": "20509:28315:尺码:M;1627207:28320:颜色分类:白色", "quantity": 200, "sku_id": "4958597356172" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:28327", "properties_name": "20509:28315:尺码:M;1627207:28327:颜色分类:酒红色", "quantity": 200, "sku_id": "4958597356175" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:3232479", "properties_name": "20509:28315:尺码:M;1627207:3232479:颜色分类:深紫色", "quantity": 200, "sku_id": "4958597356178" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:28341", "properties_name": "20509:28315:尺码:M;1627207:28341:颜色分类:黑色", "quantity": 200, "sku_id": "4958597356181" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:28335", "properties_name": "20509:28315:尺码:M;1627207:28335:颜色分类:绿色", "quantity": 200, "sku_id": "5136958387149" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:30155", "properties_name": "20509:28315:尺码:M;1627207:30155:颜色分类:杏色", "quantity": 200, "sku_id": "4964918757568" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28315;1627207:30226", "properties_name": "20509:28315:尺码:M;1627207:30226:颜色分类:浅粉色", "quantity": 200, "sku_id": "5150786423328" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:28320", "properties_name": "20509:28316:尺码:L;1627207:28320:颜色分类:白色", "quantity": 200, "sku_id": "4958597356173" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:28327", "properties_name": "20509:28316:尺码:L;1627207:28327:颜色分类:酒红色", "quantity": 200, "sku_id": "4958597356176" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:3232479", "properties_name": "20509:28316:尺码:L;1627207:3232479:颜色分类:深紫色", "quantity": 200, "sku_id": "4958597356179" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:28341", "properties_name": "20509:28316:尺码:L;1627207:28341:颜色分类:黑色", "quantity": 200, "sku_id": "4958597356182" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:28335", "properties_name": "20509:28316:尺码:L;1627207:28335:颜色分类:绿色", "quantity": 200, "sku_id": "5136958387150" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:30155", "properties_name": "20509:28316:尺码:L;1627207:30155:颜色分类:杏色", "quantity": 200, "sku_id": "4964918757569" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28316;1627207:30226", "properties_name": "20509:28316:尺码:L;1627207:30226:颜色分类:浅粉色", "quantity": 200, "sku_id": "5150786423329" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:28320", "properties_name": "20509:28317:尺码:XL;1627207:28320:颜色分类:白色", "quantity": 200, "sku_id": "4964551985973" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:28327", "properties_name": "20509:28317:尺码:XL;1627207:28327:颜色分类:酒红色", "quantity": 200, "sku_id": "4964551985975" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:3232479", "properties_name": "20509:28317:尺码:XL;1627207:3232479:颜色分类:深紫色", "quantity": 200, "sku_id": "4964551985977" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:28341", "properties_name": "20509:28317:尺码:XL;1627207:28341:颜色分类:黑色", "quantity": 200, "sku_id": "4964551985976" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:28335", "properties_name": "20509:28317:尺码:XL;1627207:28335:颜色分类:绿色", "quantity": 200, "sku_id": "5136958387151" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:30155", "properties_name": "20509:28317:尺码:XL;1627207:30155:颜色分类:杏色", "quantity": 200, "sku_id": "4964918757570" }, { "price": 158, "total_price": 0, "orginal_price": 158, "properties": "20509:28317;1627207:30226", "properties_name": "20509:28317:尺码:XL;1627207:30226:颜色分类:浅粉色", "quantity": 200, "sku_id": "5150786423330" } ] }, "seller_id": "870481044", "sales": 1000, "shop_id": "110507099", "props_list": { "20509:28314": "尺码:S", "20509:28315": "尺码:M", "20509:28316": "尺码:L", "20509:28317": "尺码:XL", "1627207:28320": "颜色分类:白色", "1627207:28327": "颜色分类:酒红色", "1627207:3232479": "颜色分类:深紫色", "1627207:28341": "颜色分类:黑色", "1627207:28335": "颜色分类:绿色", "1627207:30155": "颜色分类:杏色", "1627207:30226": "颜色分类:浅粉色" }, "seller_info": { "nick": "1112tx", "item_score": 4.77514, "score_p": 4.81006, "delivery_score": 4.83294, "shop_type": "", "user_num_id": "870481044", "sid": "110507099", "title": "", "zhuy": "https://shop110507099.taobao.com", "cert": null, "open_time": "9年老店", "credit_score": "tb-rank-crown:2", "shop_name": "SEE YOU TOMORROW" }, "tmall": false, "error": "", "location": "广东深圳", "data_from": "Ha", "has_discount": "false", "is_promotion": "false", "promo_type": null, "props_img": { "1627207:28320": "//gd3.alicdn.com/imgextra/i4/870481044/O1CN01kc82x31JaCJJupFIy_!!870481044.jpg", "1627207:28327": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01t1uYxH1JaCJCYfNBn_!!870481044.jpg", "1627207:3232479": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01Hy2bEg1JaCJ3euABW_!!870481044.jpg", "1627207:28341": "//gd3.alicdn.com/imgextra/i2/870481044/O1CN01W3PfUh1JaCJFhNE43_!!870481044.jpg", "1627207:28335": "//gd4.alicdn.com/imgextra/i2/870481044/O1CN01r6sBzb1JaCJMajq8y_!!870481044.jpg", "1627207:30155": "//gd4.alicdn.com/imgextra/i3/870481044/O1CN01MgARg51JaCJKma7hO_!!870481044.jpg", "1627207:30226": "//gd2.alicdn.com/imgextra/i1/870481044/O1CN01nreksE1JaCJmGgaY4_!!870481044.jpg" }, "format_check": "ok", "desc_img": [ "http://img.alicdn.com/imgextra/i4/870481044/O1CN01a0DbR71JaCM2XV3Y3_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01pPm6Iv1JaCLaSZl2w_!!870481044.jpg", "http://img.alicdn.com/imgextra/i3/870481044/O1CN01Nr0Tdv1JaCJCYLA3c_!!870481044.gif", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01i8wUET1JaCJKx5w4t_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01Mttxra1JaCJGav13f_!!870481044.jpg", "http://img.alicdn.com/imgextra/i3/870481044/O1CN0165KAmi1JaCJEiYbp0_!!870481044.jpg", "http://img.alicdn.com/imgextra/i3/870481044/O1CN01yO0uW41JaCJATADqT_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01m3vYox1JaCKkfmnff_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01MD3psB1JaCJsgaIRl_!!870481044.jpg", "http://img.alicdn.com/imgextra/i3/870481044/O1CN01vo3UXH1JaCJqGGdCn_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN011eNZnC1JaCJKsGHrs_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01CSoJQu1JaCJPbfFCg_!!870481044.jpg", "http://img.alicdn.com/imgextra/i3/870481044/O1CN01Pu53uc1JaCJJllu1h_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01IHwzDv1JaCJGWCLsF_!!870481044.jpg", "http://img.alicdn.com/imgextra/i1/870481044/O1CN01JfXRK21JaCJHG4ASe_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01vVxjgg1JaCJAThihP_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN01l7875s1JaCJJlmm8M_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN01af3eH01JaCJJLlj6r_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN01eJo1x11JaCJGWDMHT_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN016RyiTn1JaCJKsHxrZ_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01BuQuRb1JaCJJPM0wJ_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01CeSlXu1JaCJFAqLJS_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN01CotmK51JaCJLICxqi_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN01GIzmgP1JaCJJLkzO6_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01BiNWYi1JaCJNu8SDs_!!870481044.jpg", "http://img.alicdn.com/imgextra/i2/870481044/O1CN01oAxHNi1JaCJLIB1Hi_!!870481044.jpg", "http://img.alicdn.com/imgextra/i4/870481044/O1CN015SHHl71JaCJGWEIY8_!!870481044.jpg" ], "shop_item": [], "relate_items": [] },

你可能感兴趣的:(电商平台api,python,爬虫,开发语言)