Python爬取个人博客信息汇总

效果预览

爬取自己所写的每一条博客的标题、链接等信息：
1 Python爬取上交所年报并下载 https://blog.csdn.net/weixin_43636302/article/details/93202006
2 Excel宏的录制 https://blog.csdn.net/weixin_43636302/article/details/102981249
3 Python入门教程 https://blog.csdn.net/weixin_43636302/article/details/102981768
4 Python合并多个Excel，汇总内容到一个文件 https://blog.csdn.net/weixin_43636302/article/details/102990771
5 Python爬取最近上映的电影评论并生成词云——决战中途岛 https://blog.csdn.net/weixin_43636302/article/details/102992231
6 Python画图包matplotlib——散点图、折线图的绘制 https://blog.csdn.net/weixin_43636302/article/details/102993593
7 Python绘制心形图案 https://blog.csdn.net/weixin_43636302/article/details/102993691
8 Python画五角星层叠图案和分形树——turtle包的使用 https://blog.csdn.net/weixin_43636302/article/details/102993726
9 Python爬取上交所市值排名——根据输入的日期区间导出结果到Excel https://blog.csdn.net/weixin_43636302/article/details/102993915
10 AE入门案例——制作新闻联播片头 https://blog.csdn.net/weixin_43636302/article/details/102995615
11 PS图层蒙版应用——图片抠字 https://blog.csdn.net/weixin_43636302/article/details/103001265
12 LaTeX入门 https://blog.csdn.net/weixin_43636302/article/details/103007370
13 Excel建立个人宏工作簿 https://blog.csdn.net/weixin_43636302/article/details/103012880
14 Stata12文件转码为Stata15格式解决办法 https://blog.csdn.net/weixin_43636302/article/details/103013804
15 Stata进行矩阵运算，求逆矩阵、特征根、特征向量 https://blog.csdn.net/weixin_43636302/article/details/103016315
16 Stata循环语句的运用——循环数值、变量、文件 https://blog.csdn.net/weixin_43636302/article/details/103021245
17 Stata画图——散点图与折线图 https://blog.csdn.net/weixin_43636302/article/details/103021293
18 Stata计算可操纵性应计利润——基于琼斯模型 https://blog.csdn.net/weixin_43636302/article/details/103021334
19 Stata安装外部命令——以Winsor2缩尾命令为例 https://blog.csdn.net/weixin_43636302/article/details/103021340
20 Stata结果输出常用命令 https://blog.csdn.net/weixin_43636302/article/details/103021348
21 Stata导入数据，批量贴标签 https://blog.csdn.net/weixin_43636302/article/details/103022689
22 Stata进阶用法——文件横向合并，纵向合并 https://blog.csdn.net/weixin_43636302/article/details/103024014
23 Stata字符型与数值型数据的转换——上市公司证券代码字符型与数值型的互换 https://blog.csdn.net/weixin_43636302/article/details/103024588
24 Excel中常用快捷键及小技巧 https://blog.csdn.net/weixin_43636302/article/details/103030905
25 Scrapy框架爬取名人名言网信息——基于css内置选择器 https://blog.csdn.net/weixin_43636302/article/details/103038430
26 Scrapy框架爬取dmoztools网站两个网址信息——基于Xpath内置选择器 https://blog.csdn.net/weixin_43636302/article/details/103040563
27 Excel宏的基本逻辑语句 https://blog.csdn.net/weixin_43636302/article/details/103049127
28 Excel弹出宏对话框激活工作表 https://blog.csdn.net/weixin_43636302/article/details/103050748
29 Excel宏批量转置并删除空格 https://blog.csdn.net/weixin_43636302/article/details/103054650
30 Python中用Requests爬取名人名言网信息——基于BeautifulSoup和CSS选择器 https://blog.csdn.net/weixin_43636302/article/details/103055711
31 Python自动化办公——生成Excel入库统计表 https://blog.csdn.net/weixin_43636302/article/details/103058576
32 Python中输入任意字符型起始日期，转换成日期格式并循环遍历 https://blog.csdn.net/weixin_43636302/article/details/103062493
33 Python爬虫基本流程梳理：构造请求头、发起请求、json格式数据解码转码、读写CSV文件——基于AJAX网页爬虫 https://blog.csdn.net/weixin_43636302/article/details/103062655
34 Python爬虫模块化格式 https://blog.csdn.net/weixin_43636302/article/details/103064065
35 Python常用快捷键 https://blog.csdn.net/weixin_43636302/article/details/103095466
36 Pyhton里面format的几种等价用法 https://blog.csdn.net/weixin_43636302/article/details/103095951
37 Python爬虫实例——基于Xpath爬取西刺网站ip、端口信息 https://blog.csdn.net/weixin_43636302/article/details/103096023
38 xpath helper安装和使用方法 https://blog.csdn.net/weixin_43636302/article/details/103134303
39 Python数据分析基础——numpy用法 https://blog.csdn.net/weixin_43636302/article/details/103136449
40 Python数据分析基础——pandas用法 https://blog.csdn.net/weixin_43636302/article/details/103152391
41 如何快速转载CSDN博客 https://blog.csdn.net/weixin_43636302/article/details/103189367
42 【数据处理】pandas DataFrame 对多个sheet表格的excel 读取、处理、保存 https://blog.csdn.net/weixin_43636302/article/details/103193331
43 python中使用xlrd、xlwt操作excel表格详解 https://blog.csdn.net/weixin_43636302/article/details/103198889
44 Python中pandas和xlrd、xlwt操作Excel用法的总结 https://blog.csdn.net/weixin_43636302/article/details/103199012
45 Mysql数据库入门笔记 https://blog.csdn.net/weixin_43636302/article/details/103233312
46 Win10环境下安装Mysql5.7.23的笔记心得 https://blog.csdn.net/weixin_43636302/article/details/103230356
47 chromedriver在python环境下的配置与使用 https://blog.csdn.net/weixin_43636302/article/details/103267779
48 Office2013邮件合并简单案例——批量生成导出PDF和Word https://blog.csdn.net/weixin_43636302/article/details/103349974
49 Python读写txt文件、转换csv文件与pandas条件计数、求和的综合运用 https://blog.csdn.net/weixin_43636302/article/details/103414582
50 珠算加法六百六口诀step by step https://blog.csdn.net/weixin_43636302/article/details/103448372
51 珠算小九归之二归 https://blog.csdn.net/weixin_43636302/article/details/103483806
52 珠算小九归之三归 https://blog.csdn.net/weixin_43636302/article/details/103484012
53 珠算小九归之四归 https://blog.csdn.net/weixin_43636302/article/details/103484119
54 python代码直接生成可执行exe文件——以年报下载器制作为例 https://blog.csdn.net/weixin_43636302/article/details/103555585
55 Python批量下载上交所、深交所年报或半年报并生成年报下载器exe文件 https://blog.csdn.net/weixin_43636302/article/details/103555752
56 Selenium自动化测试工具——以爬取京东商品信息为例 https://blog.csdn.net/weixin_43636302/article/details/103571335
57 Python量化交易——爬取股票日K线画图检验股票策略 https://blog.csdn.net/weixin_43636302/article/details/103580565
58 Selenium爬取会计师事务所新闻信息——以中准会计师事务所为例 https://blog.csdn.net/weixin_43636302/article/details/103593242
59 正则、bs4爬取会计师事务所新闻信息——以利安达会计师事务所为例 https://blog.csdn.net/weixin_43636302/article/details/103593931
60 Selenium自动翻页爬取证券公司公告信息——以中国平安为例 https://blog.csdn.net/weixin_43636302/article/details/103594431
61 正则爬取猫眼电影榜单信息 https://blog.csdn.net/weixin_43636302/article/details/103595240
62 Python解析PDF文本 https://blog.csdn.net/weixin_43636302/article/details/103606925
63 珠算小九归之五归 https://blog.csdn.net/weixin_43636302/article/details/103487003
64 珠算小九归之六归 https://blog.csdn.net/weixin_43636302/article/details/103490797
65 珠算小九归之七归 https://blog.csdn.net/weixin_43636302/article/details/103491556
66 珠算小九归之八归 https://blog.csdn.net/weixin_43636302/article/details/103491697
67 珠算小九归之九归 https://blog.csdn.net/weixin_43636302/article/details/103491780
68 珠算大九归之一六归 https://blog.csdn.net/weixin_43636302/article/details/103609554
69 珠算大九归之二归四除 https://blog.csdn.net/weixin_43636302/article/details/103610613
70 珠算大九归之三九归 https://blog.csdn.net/weixin_43636302/article/details/103613169
71 珠算大九归之四七归 https://blog.csdn.net/weixin_43636302/article/details/103613223
72 珠算大九归之五五归 https://blog.csdn.net/weixin_43636302/article/details/103613271
73 珠算大九归之六八归 https://blog.csdn.net/weixin_43636302/article/details/103613732
74 珠算大九归之七四归 https://blog.csdn.net/weixin_43636302/article/details/103613761
75 珠算大九归之八六归 https://blog.csdn.net/weixin_43636302/article/details/103613778
76 珠算大九归之九三归 https://blog.csdn.net/weixin_43636302/article/details/103613790

代码实现

import requests
# import re
from  bs4 import BeautifulSoup
import pandas as pd
from selenium import webdriver
# browser = webdriver.Chrome()
import time

def getData(page):
    url=f'https://blog.csdn.net/weixin_43636302/article/list/{page}'
    r=requests.get(url)
    soup = BeautifulSoup(r.text, 'lxml')
    data=soup.select('div div div h4 a')
    return data

if  __name__=='__main__':
    df = pd.DataFrame({'title':[],'link':[]})
    i=int(input('请输入爬取总页数：'))
    for page in range(1,i+1):
        data=getData(page)
        for da in data:
            title=da.get_text()[11:]
            link=da['href']
            print(title)
            print(link)
            #自动打开网页
            # browser.get(link)
            # time.sleep(2)
            new = pd.DataFrame({'title':[title],'link':[link]})
            df= df.append(new, ignore_index=True)
    #调整列的位置
    mid = df['title']
    df.drop(labels=['title'], axis=1, inplace=True)
    df.insert(0, 'title', mid)
    df.to_csv('res.csv')