通过request拿到网页原源代码
通过re来提取想要的有效信息
import requests
import re
请求头
headers = {
"User-Agent": "Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (HTML, "
"like Gecko) Chrome/90.0.4430.93 Mobile Safari/537.36 "
}
预编译所有re表达式
sub = re.compile(r''?P.*?)">.*?</a><span>(?P<time>.*?)</span></li>'</span><span class="token punctuation">,</span> re<span class="token punctuation">.</span>S<span class="token punctuation">)</span>
detail_1 <span class="token operator">=</span> re<span class="token punctuation">.</span><span class="token builtin">compile</span><span class="token punctuation">(</span>r<span class="token string">'<div id="vsb_content_501">(?P<detail>.*?)<div id="div_vote_id">'</span><span class="token punctuation">,</span> re<span class="token punctuation">.</span>S<span class="token punctuation">)</span>
detail_1 <span class="token operator">=</span> re<span class="token punctuation">.</span><span class="token builtin">compile</span><span class="token punctuation">(</span>r<span class="token string">'<img border="0".*"></p>(?P<detail>.*?)<div id="div_vote_id">'</span><span class="token punctuation">,</span> re<span class="token punctuation">.</span>S<span class="token punctuation">)</span>
</code></pre>
<h1>爬取网页源代码</h1>
<h1>遍历389页新闻目录页</h1>
<pre><code class="prism language-python"><span class="token keyword">for</span> i <span class="token keyword">in</span> <span class="token builtin">range</span><span class="token punctuation">(</span><span class="token number">389</span><span class="token punctuation">,</span> <span class="token number">0</span><span class="token punctuation">,</span> <span class="token operator">-</span><span class="token number">1</span><span class="token punctuation">)</span><span class="token punctuation">:</span>
</code></pre>
<p>在所写文档中将每页区内容区分开</p>
<pre><code class="prism language-python"> news <span class="token operator">=</span> <span class="token builtin">open</span><span class="token punctuation">(</span><span class="token string">"科院新闻.txt"</span><span class="token punctuation">,</span> <span class="token string">"a"</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"*"</span><span class="token operator">*</span><span class="token number">10</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"第"</span><span class="token operator">+</span><span class="token builtin">str</span><span class="token punctuation">(</span><span class="token number">390</span><span class="token operator">-</span>i<span class="token punctuation">)</span><span class="token operator">+</span><span class="token string">"页"</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"*"</span><span class="token operator">*</span><span class="token number">10</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"\n"</span><span class="token operator">*</span><span class="token number">2</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>close<span class="token punctuation">(</span><span class="token punctuation">)</span>
</code></pre>
<p>在控制台输出进度</p>
<pre><code class="prism language-python"><span class="token keyword">print</span><span class="token punctuation">(</span><span class="token string">"第"</span><span class="token operator">+</span><span class="token builtin">str</span><span class="token punctuation">(</span><span class="token number">390</span><span class="token operator">-</span>i<span class="token punctuation">)</span><span class="token operator">+</span><span class="token string">"页新闻下载中"</span><span class="token punctuation">)</span>
</code></pre>
<p>获取每一页的源代码</p>
<pre><code class="prism language-python">url <span class="token operator">=</span> <span class="token string">"https://news.hist.edu.cn/kyyw/"</span><span class="token operator">+</span><span class="token builtin">str</span><span class="token punctuation">(</span>i<span class="token punctuation">)</span><span class="token operator">+</span><span class="token string">".htm"</span>
resp <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>url<span class="token punctuation">,</span> headers<span class="token operator">=</span>headers<span class="token punctuation">)</span>
content <span class="token operator">=</span> resp<span class="token punctuation">.</span>content<span class="token punctuation">.</span>decode<span class="token punctuation">(</span><span class="token punctuation">)</span>
</code></pre>
<p>匹配所需内容</p>
<pre><code class="prism language-python"> result <span class="token operator">=</span> sub<span class="token punctuation">.</span>finditer<span class="token punctuation">(</span>content<span class="token punctuation">)</span>
</code></pre>
<p>调出并使用匹配到的内容</p>
<pre><code class="prism language-python"><span class="token keyword">for</span> it <span class="token keyword">in</span> result<span class="token punctuation">:</span>
title <span class="token operator">=</span> it<span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token string">"title"</span><span class="token punctuation">)</span>
time <span class="token operator">=</span> it<span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token string">"time"</span><span class="token punctuation">)</span>
half <span class="token operator">=</span> it<span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token string">"content_url"</span><span class="token punctuation">)</span>
</code></pre>
<p>拼出接下来要前往的网页</p>
<pre><code class="prism language-python">whole <span class="token operator">=</span> <span class="token string">"https://news.hist.edu.cn/"</span> <span class="token operator">+</span> half
</code></pre>
<p>爬取新的网页的源代码</p>
<pre><code class="prism language-python">article <span class="token operator">=</span> requests<span class="token punctuation">.</span>get<span class="token punctuation">(</span>whole<span class="token punctuation">,</span> headers<span class="token operator">=</span>headers<span class="token punctuation">)</span>
detail_2 <span class="token operator">=</span> article<span class="token punctuation">.</span>content<span class="token punctuation">.</span>decode<span class="token punctuation">(</span><span class="token punctuation">)</span>
</code></pre>
<p>匹配所需内容</p>
<pre><code class="prism language-python">result2 <span class="token operator">=</span> detail_1<span class="token punctuation">.</span>finditer<span class="token punctuation">(</span>detail_2<span class="token punctuation">)</span>
</code></pre>
<p>创建科院新闻txt文档，<br> a：打开一个文件用于追加</p>
<pre><code class="prism language-python">news <span class="token operator">=</span> <span class="token builtin">open</span><span class="token punctuation">(</span><span class="token string">"科院新闻.txt"</span><span class="token punctuation">,</span> <span class="token string">"a"</span><span class="token punctuation">,</span> encoding<span class="token operator">=</span><span class="token string">'utf-8'</span><span class="token punctuation">)</span>
</code></pre>
<p>调出并处理匹配到的内容</p>
<pre><code class="prism language-python"> <span class="token keyword">for</span> mor <span class="token keyword">in</span> result2<span class="token punctuation">:</span>
content <span class="token operator">=</span> mor<span class="token punctuation">.</span>group<span class="token punctuation">(</span><span class="token string">"detail"</span><span class="token punctuation">)</span>
</code></pre>
<p>进行数据清洗</p>
<pre><code class="prism language-python"> content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'(0, 0, 0)'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'&'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'<'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'>'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'/'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'a'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'b'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'c'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'d'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'e'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'f'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'g'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'h'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'i'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'j'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'k'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'b'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'l'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'m'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'n'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'o'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'p'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'q'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'r'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'s'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'t'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'u'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'v'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'w'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'x'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'y'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'z'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'=":'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">';-:'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'=":'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">': "'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'="-'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'="_"'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
content <span class="token operator">=</span> content<span class="token punctuation">.</span>replace<span class="token punctuation">(</span><span class="token string">'\r\n'</span><span class="token punctuation">,</span> <span class="token string">''</span><span class="token punctuation">)</span>
</code></pre>
<p>写入标题，内容，时间</p>
<pre><code class="prism language-python"> news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"《"</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span>title<span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"》"</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"\n"</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span>content<span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"\n"</span><span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span>time<span class="token punctuation">)</span>
news<span class="token punctuation">.</span>write<span class="token punctuation">(</span><span class="token string">"\n"</span> <span class="token operator">*</span> <span class="token number">3</span><span class="token punctuation">)</span>
</code></pre>
<p>当我们写文件时，操作系统往往不会立刻把数据写入磁盘，而是放到内存缓存起来，空闲的时候再慢慢写入。<br> 只有调用close()方法时，操作系统才保证把没有写入的数据全部写入磁盘。忘记调用close()的后果是数据可能只写了一部分到磁盘，剩下的丢失了。</p>
<pre><code class="prism language-python"> news<span class="token punctuation">.</span>close<span class="token punctuation">(</span><span class="token punctuation">)</span>
</code></pre>
<blockquote>
<p>自己看完视频敲得，有很多操作可能有些沙雕，欢迎大佬在评论区指手画脚</p>
</blockquote>
</div>
</div>
</div>
</div>
</div>
<!--PC和WAP自适应版-->
<div id="SOHUCS" sid="1450730264052449280"></div>
<script type="text/javascript" src="/views/front/js/chanyan.js"></script>
<!-- 文章页-底部 动态广告位 -->
<div class="youdao-fixed-ad" id="detail_ad_bottom"></div>
</div>
<div class="col-md-3">
<div class="row" id="ad">
<!-- 文章页-右侧1 动态广告位 -->
<div id="right-1" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_1"> </div>
</div>
<!-- 文章页-右侧2 动态广告位 -->
<div id="right-2" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_2"></div>
</div>
<!-- 文章页-右侧3 动态广告位 -->
<div id="right-3" class="col-lg-12 col-md-12 col-sm-4 col-xs-4 ad">
<div class="youdao-fixed-ad" id="detail_ad_3"></div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="container">
<h4 class="pt20 mb15 mt0 border-top">你可能感兴趣的:(python)</h4>
<div id="paradigm-article-related">
<div class="recommend-post mb30">
<ul class="widget-links">
<li><a href="/article/1835511912843014144.htm"
title="理解Gunicorn：Python WSGI服务器的基石" target="_blank">理解Gunicorn：Python WSGI服务器的基石</a>
<span class="text-muted">范范0825</span>
<a class="tag" taget="_blank" href="/search/ipython/1.htm">ipython</a><a class="tag" taget="_blank" href="/search/linux/1.htm">linux</a><a class="tag" taget="_blank" href="/search/%E8%BF%90%E7%BB%B4/1.htm">运维</a>
<div>理解Gunicorn：PythonWSGI服务器的基石介绍Gunicorn，全称GreenUnicorn，是一个为PythonWSGI（WebServerGatewayInterface）应用设计的高效、轻量级HTTP服务器。作为PythonWeb应用部署的常用工具，Gunicorn以其高性能和易用性著称。本文将介绍Gunicorn的基本概念、安装和配置，帮助初学者快速上手。1.什么是Gunico</div>
</li>
<li><a href="/article/1835506869838376960.htm"
title="Python数据分析与可视化实战指南" target="_blank">Python数据分析与可视化实战指南</a>
<span class="text-muted">William数据分析</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE/1.htm">数据</a>
<div>在数据驱动的时代，Python因其简洁的语法、强大的库生态系统以及活跃的社区，成为了数据分析与可视化的首选语言。本文将通过一个详细的案例，带领大家学习如何使用Python进行数据分析，并通过可视化来直观呈现分析结果。一、环境准备1.1安装必要库在开始数据分析和可视化之前，我们需要安装一些常用的库。主要包括pandas、numpy、matplotlib和seaborn等。这些库分别用于数据处理、数学</div>
</li>
<li><a href="/article/1835505858939809792.htm"
title="python os.environ" target="_blank">python os.environ</a>
<span class="text-muted">江湖偌大</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%B7%B1%E5%BA%A6%E5%AD%A6%E4%B9%A0/1.htm">深度学习</a>
<div>os.environ['TF_CPP_MIN_LOG_LEVEL']='0'#默认值，输出所有信息os.environ['TF_CPP_MIN_LOG_LEVEL']='1'#屏蔽通知信息（INFO）os.environ['TF_CPP_MIN_LOG_LEVEL']='2'#屏蔽通知信息和警告信息（INFO\WARNING）os.environ['TF_CPP_MIN_LOG_LEVEL']='</div>
</li>
<li><a href="/article/1835505606245576704.htm"
title="Python中os.environ基本介绍及使用方法" target="_blank">Python中os.environ基本介绍及使用方法</a>
<span class="text-muted">鹤冲天Pro</span>
<a class="tag" taget="_blank" href="/search/%23/1.htm">#</a><a class="tag" taget="_blank" href="/search/Python/1.htm">Python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%9C%8D%E5%8A%A1%E5%99%A8/1.htm">服务器</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>文章目录python中os.environos.environ简介os.environ进行环境变量的增删改查python中os.environ的使用详解1.简介2.key字段详解2.1常见key字段3.os.environ.get()用法4.环境变量的增删改查和判断是否存在4.1新增环境变量4.2更新环境变量4.3获取环境变量4.4删除环境变量4.5判断环境变量是否存在python中os.envi</div>
</li>
<li><a href="/article/1835505226933694464.htm"
title="Pyecharts数据可视化大屏：打造沉浸式数据分析体验" target="_blank">Pyecharts数据可视化大屏：打造沉浸式数据分析体验</a>
<span class="text-muted">我的运维人生</span>
<a class="tag" taget="_blank" href="/search/%E4%BF%A1%E6%81%AF%E5%8F%AF%E8%A7%86%E5%8C%96/1.htm">信息可视化</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/1.htm">数据分析</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E6%8C%96%E6%8E%98/1.htm">数据挖掘</a><a class="tag" taget="_blank" href="/search/%E8%BF%90%E7%BB%B4%E5%BC%80%E5%8F%91/1.htm">运维开发</a><a class="tag" taget="_blank" href="/search/%E6%8A%80%E6%9C%AF%E5%85%B1%E4%BA%AB/1.htm">技术共享</a>
<div>Pyecharts数据可视化大屏：打造沉浸式数据分析体验在当今这个数据驱动的时代，如何将海量数据以直观、生动的方式展现出来，成为了数据分析师和企业决策者关注的焦点。Pyecharts，作为一款基于Python的开源数据可视化库，凭借其丰富的图表类型、灵活的配置选项以及高度的定制化能力，成为了构建数据可视化大屏的理想选择。本文将深入探讨如何利用Pyecharts打造数据可视化大屏，并通过实际代码案例</div>
</li>
<li><a href="/article/1835504217729626112.htm"
title="Python教程：一文了解使用Python处理XPath" target="_blank">Python教程：一文了解使用Python处理XPath</a>
<span class="text-muted">旦莫</span>
<a class="tag" taget="_blank" href="/search/Python%E8%BF%9B%E9%98%B6/1.htm">Python进阶</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>目录1.环境准备1.1安装lxml1.2验证安装2.XPath基础2.1什么是XPath？2.2XPath语法2.3示例XML文档3.使用lxml解析XML3.1解析XML文档3.2查看解析结果4.XPath查询4.1基本路径查询4.2使用属性查询4.3查询多个节点5.XPath的高级用法5.1使用逻辑运算符5.2使用函数6.实战案例6.1从网页抓取数据6.1.1安装Requests库6.1.2代</div>
</li>
<li><a href="/article/1835503965563875328.htm"
title="python os.environ_python os.environ 读取和设置环境变量" target="_blank">python os.environ_python os.environ 读取和设置环境变量</a>
<span class="text-muted">weixin_39605414</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/os.environ/1.htm">os.environ</a>
<div>>>>importos>>>os.environ.keys()['LC_NUMERIC','GOPATH','GOROOT','GOBIN','LESSOPEN','SSH_CLIENT','LOGNAME','USER','HOME','LC_PAPER','PATH','DISPLAY','LANG','TERM','SHELL','J2REDIR','LC_MONETARY','QT_QPA</div>
</li>
<li><a href="/article/1835497664922349568.htm"
title="使用Faiss进行高效相似度搜索" target="_blank">使用Faiss进行高效相似度搜索</a>
<span class="text-muted">llzwxh888</span>
<a class="tag" taget="_blank" href="/search/faiss/1.htm">faiss</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a>
<div>在现代AI应用中，快速和高效的相似度搜索是至关重要的。Faiss（FacebookAISimilaritySearch）是一个专门用于快速相似度搜索和聚类的库，特别适用于高维向量。本文将介绍如何使用Faiss来进行相似度搜索，并结合Python代码演示其基本用法。什么是Faiss？Faiss是一个由FacebookAIResearch团队开发的开源库，主要用于高维向量的相似性搜索和聚类。Faiss</div>
</li>
<li><a href="/article/1835497665853485056.htm"
title="python是什么意思中文-在python中%是什么意思" target="_blank">python是什么意思中文-在python中%是什么意思</a>
<span class="text-muted">编程大乐趣</span>
<div>Python中%有两种：1、数值运算：%代表取模，返回除法的余数。如：>>>7%212、%操作符（字符串格式化，stringformatting），说明如下：%[(name)][flags][width].[precision]typecode(name)为命名flags可以有+，-，''或0。+表示右对齐。-表示左对齐。''为一个空格，表示在正数的左侧填充一个空格，从而与负数对齐。0表示使用0填</div>
</li>
<li><a href="/article/1835495644123459584.htm"
title="Day1笔记-Python简介&标识符和关键字&输入输出" target="_blank">Day1笔记-Python简介&标识符和关键字&输入输出</a>
<span class="text-muted">~在杰难逃~</span>
<a class="tag" taget="_blank" href="/search/Python/1.htm">Python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%95%B0%E6%8D%AE/1.htm">大数据</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/1.htm">数据分析</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E6%8C%96%E6%8E%98/1.htm">数据挖掘</a>
<div>大家好，从今天开始呢，杰哥开展一个新的专栏，当然，数据分析部分也会不定时更新的，这个新的专栏主要是讲解一些Python的基础语法和知识，帮助0基础的小伙伴入门和学习Python，感兴趣的小伙伴可以开始认真学习啦！一、Python简介【了解】1.计算机工作原理编程语言就是用来定义计算机程序的形式语言。我们通过编程语言来编写程序代码，再通过语言处理程序执行向计算机发送指令，让计算机完成对应的工作，编程</div>
</li>
<li><a href="/article/1835495517774245888.htm"
title="python八股文面试题分享及解析(1)" target="_blank">python八股文面试题分享及解析(1)</a>
<span class="text-muted">Shawn________</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a>
<div>#1.'''a=1b=2不用中间变量交换a和b'''#1.a=1b=2a,b=b,aprint(a)print(b)结果：21#2.ll=[]foriinrange(3):ll.append({'num':i})print(11)结果:#[{'num':0},{'num':1},{'num':2}]#3.kk=[]a={'num':0}foriinrange(3):#0,12#可变类型，不仅仅改变</div>
</li>
<li><a href="/article/1835493753557708800.htm"
title="每日算法&面试题，大厂特训二十八天——第二十天（树）" target="_blank">每日算法&面试题，大厂特训二十八天——第二十天（树）</a>
<span class="text-muted">肥学</span>
<a class="tag" taget="_blank" href="/search/%E2%9A%A1%E7%AE%97%E6%B3%95%E9%A2%98%E2%9A%A1%E9%9D%A2%E8%AF%95%E9%A2%98%E6%AF%8F%E6%97%A5%E7%B2%BE%E8%BF%9B/1.htm">⚡算法题⚡面试题每日精进</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84/1.htm">数据结构</a>
<div>目录标题导读算法特训二十八天面试题点击直接资料领取导读肥友们为了更好的去帮助新同学适应算法和面试题，最近我们开始进行专项突击一步一步来。上一期我们完成了动态规划二十一天现在我们进行下一项对各类算法进行二十八天的一个小总结。还在等什么快来一起肥学进行二十八天挑战吧！！特别介绍小白练手专栏，适合刚入手的新人欢迎订阅编程小白进阶python有趣练手项目里面包括了像《机器人尬聊》《恶搞程序》这样的有趣文章</div>
</li>
<li><a href="/article/1835493626688401408.htm"
title="Python快速入门 —— 第三节：类与对象" target="_blank">Python快速入门 —— 第三节：类与对象</a>
<span class="text-muted">孤华暗香</span>
<a class="tag" taget="_blank" href="/search/Python%E5%BF%AB%E9%80%9F%E5%85%A5%E9%97%A8/1.htm">Python快速入门</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>第三节：类与对象目标：了解面向对象编程的基础概念，并学会如何定义类和创建对象。内容：类与对象：定义类：class关键字。类的构造函数：__init__()。类的属性和方法。对象的创建与使用。示例：classStudent:def__init__(self,name,age,major):self.name&#</div>
</li>
<li><a href="/article/1835492869062881280.htm"
title="pyecharts——绘制柱形图折线图" target="_blank">pyecharts——绘制柱形图折线图</a>
<span class="text-muted">2224070247</span>
<a class="tag" taget="_blank" href="/search/%E4%BF%A1%E6%81%AF%E5%8F%AF%E8%A7%86%E5%8C%96/1.htm">信息可视化</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%8F%AF%E8%A7%86%E5%8C%96/1.htm">数据可视化</a>
<div>一、pyecharts概述自2013年6月百度EFE(ExcellentFrontEnd）数据可视化团队研发的ECharts1.0发布到GitHub网站以来，ECharts一直备受业界权威的关注并获得广泛好评，成为目前成熟且流行的数据可视化图表工具，被应用到诸多数据可视化的开发领域。Python作为数据分析领域最受欢迎的语言，也加入ECharts的使用行列，并研发出方便Python开发者使用的数据</div>
</li>
<li><a href="/article/1835491859351302144.htm"
title="Python 实现图片裁剪（附代码） | Python工具" target="_blank">Python 实现图片裁剪（附代码） | Python工具</a>
<span class="text-muted">剑客阿良_ALiang</span>
<div>前言本文提供将图片按照自定义尺寸进行裁剪的工具方法，一如既往的实用主义。环境依赖ffmpeg环境安装，可以参考我的另一篇文章：windowsffmpeg安装部署_阿良的博客-CSDN博客本文主要使用到的不是ffmpeg，而是ffprobe也在上面这篇文章中的zip包中。ffmpy安装：pipinstallffmpy-ihttps://pypi.douban.com/simple代码不废话了，上代码</div>
</li>
<li><a href="/article/1835491353451130880.htm"
title="【华为OD技术面试真题 - 技术面】- python八股文真题题库（4)" target="_blank">【华为OD技术面试真题 - 技术面】- python八股文真题题库（4)</a>
<span class="text-muted">算法大师</span>
<a class="tag" taget="_blank" href="/search/%E5%8D%8E%E4%B8%BAod/1.htm">华为od</a><a class="tag" taget="_blank" href="/search/%E9%9D%A2%E8%AF%95/1.htm">面试</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a>
<div>华为OD面试真题精选专栏：华为OD面试真题精选目录:2024华为OD面试手撕代码真题目录以及八股文真题目录文章目录华为OD面试真题精选**1.Python中的`with`**用途和功能自动资源管理示例：文件操作上下文管理协议示例代码工作流程解析优点2.\_\_new\_\_和**\_\_init\_\_**区别__new____init__区别总结3.**切片（Slicing）操作**基本切片语法</div>
</li>
<li><a href="/article/1835490974911000576.htm"
title="python os 环境变量" target="_blank">python os 环境变量</a>
<span class="text-muted">CV矿工</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a><a class="tag" taget="_blank" href="/search/numpy/1.htm">numpy</a>
<div>环境变量：环境变量是程序和操作系统之间的通信方式。有些字符不宜明文写进代码里，比如数据库密码，个人账户密码，如果写进自己本机的环境变量里，程序用的时候通过os.environ.get（）取出来就行了。os.environ是一个环境变量的字典。环境变量的相关操作importos"""设置/修改环境变量：os.environ[‘环境变量名称’]=‘环境变量值’#其中key和value均为string类</div>
</li>
<li><a href="/article/1835490218845761536.htm"
title="Python爬虫解析工具之xpath使用详解" target="_blank">Python爬虫解析工具之xpath使用详解</a>
<span class="text-muted">eqa11</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E7%88%AC%E8%99%AB/1.htm">爬虫</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>文章目录Python爬虫解析工具之xpath使用详解一、引言二、环境准备1、插件安装2、依赖库安装三、xpath语法详解1、路径表达式2、通配符3、谓语4、常用函数四、xpath在Python代码中的使用1、文档树的创建2、使用xpath表达式3、获取元素内容和属性五、总结Python爬虫解析工具之xpath使用详解一、引言在Python爬虫开发中，数据提取是一个至关重要的环节。xpath作为一门</div>
</li>
<li><a href="/article/1835483915071090688.htm"
title="【华为OD技术面试真题 - 技术面】- python八股文真题题库（1）" target="_blank">【华为OD技术面试真题 - 技术面】- python八股文真题题库（1）</a>
<span class="text-muted">算法大师</span>
<a class="tag" taget="_blank" href="/search/%E5%8D%8E%E4%B8%BAod/1.htm">华为od</a><a class="tag" taget="_blank" href="/search/%E9%9D%A2%E8%AF%95/1.htm">面试</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a>
<div>华为OD面试真题精选专栏：华为OD面试真题精选目录:2024华为OD面试手撕代码真题目录以及八股文真题目录文章目录华为OD面试真题精选1.数据预处理流程数据预处理的主要步骤工具和库2.介绍线性回归、逻辑回归模型线性回归（LinearRegression）模型形式：关键点：逻辑回归（LogisticRegression）模型形式：关键点：参数估计与评估：3.python浅拷贝及深拷贝浅拷贝（Shal</div>
</li>
<li><a href="/article/1835483159630802944.htm"
title="nosql数据库技术与应用知识点" target="_blank">nosql数据库技术与应用知识点</a>
<span class="text-muted">皆过客，揽星河</span>
<a class="tag" taget="_blank" href="/search/NoSQL/1.htm">NoSQL</a><a class="tag" taget="_blank" href="/search/nosql/1.htm">nosql</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%BA%93/1.htm">数据库</a><a class="tag" taget="_blank" href="/search/%E5%A4%A7%E6%95%B0%E6%8D%AE/1.htm">大数据</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/1.htm">数据分析</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84/1.htm">数据结构</a><a class="tag" taget="_blank" href="/search/%E9%9D%9E%E5%85%B3%E7%B3%BB%E5%9E%8B%E6%95%B0%E6%8D%AE%E5%BA%93/1.htm">非关系型数据库</a>
<div>Nosql知识回顾大数据处理流程数据采集(flume、爬虫、传感器)数据存储(本门课程NoSQL所处的阶段)Hdfs、MongoDB、HBase等数据清洗(入仓)Hive等数据处理、分析(Spark、Flink等)数据可视化数据挖掘、机器学习应用(Python、SparkMLlib等)大数据时代存储的挑战(三高)高并发(同一时间很多人访问)高扩展(要求随时根据需求扩展存储)高效率(要求读写速度快)</div>
</li>
<li><a href="/article/1835481269690003456.htm"
title="《Python数据分析实战终极指南》" target="_blank">《Python数据分析实战终极指南》</a>
<span class="text-muted">xjt921122</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E5%88%86%E6%9E%90/1.htm">数据分析</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>对于分析师来说，大家在学习Python数据分析的路上，多多少少都遇到过很多大坑**，有关于技能和思维的**：Excel已经没办法处理现有的数据量了，应该学Python吗？找了一大堆Python和Pandas的资料来学习，为什么自己动手就懵了？跟着比赛类公开数据分析案例练了很久，为什么当自己面对数据需求还是只会数据处理而没有分析思路？学了对比、细分、聚类分析，也会用PEST、波特五力这类分析法，为啥</div>
</li>
<li><a href="/article/1835477362700021760.htm"
title="Python中深拷贝与浅拷贝的区别" target="_blank">Python中深拷贝与浅拷贝的区别</a>
<span class="text-muted">yuxiaoyu.</span>
<div>转自：http://blog.csdn.net/u014745194/article/details/70271868定义：在Python中对象的赋值其实就是对象的引用。当创建一个对象，把它赋值给另一个变量的时候，python并没有拷贝这个对象，只是拷贝了这个对象的引用而已。浅拷贝：拷贝了最外围的对象本身，内部的元素都只是拷贝了一个引用而已。也就是，把对象复制一遍，但是该对象中引用的其他对象我不复</div>
</li>
<li><a href="/article/1835476983614631936.htm"
title="Python开发常用的三方模块如下：" target="_blank">Python开发常用的三方模块如下：</a>
<span class="text-muted">换个网名有点难</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>Python是一门功能强大的编程语言，拥有丰富的第三方库，这些库为开发者提供了极大的便利。以下是100个常用的Python库，涵盖了多个领域：1、NumPy，用于科学计算的基础库。2、Pandas，提供数据结构和数据分析工具。3、Matplotlib，一个绘图库。4、Scikit-learn，机器学习库。5、SciPy，用于数学、科学和工程的库。6、TensorFlow，由Google开发的开源机</div>
</li>
<li><a href="/article/1835473704432267264.htm"
title="Python编译器" target="_blank">Python编译器</a>
<span class="text-muted">鹿鹿~</span>
<a class="tag" taget="_blank" href="/search/Python%E7%BC%96%E8%AF%91%E5%99%A8/1.htm">Python编译器</a><a class="tag" taget="_blank" href="/search/Python/1.htm">Python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a><a class="tag" taget="_blank" href="/search/%E5%90%8E%E7%AB%AF/1.htm">后端</a>
<div>嘿嘿嘿我又来了啊有些小盆友可能不知道Python其实是有编译器的，也就是PyCharm。你们可能会问到这个是干嘛的又不可以吃也不可以穿好像没有什么用，其实你还说对了这个还真的不可以吃也不可以穿，但是它用来干嘛的呢。用来编译你所打出的代码进行运行（可能这里说的有点不对但是只是个人认为）现在我们来说说PyCharm是用来干嘛的。PyCharm是一种PythonIDE，带有一整套可以帮助用户在使用Pyt</div>
</li>
<li><a href="/article/1835471437754888192.htm"
title="一文掌握python面向对象魔术方法（二）" target="_blank">一文掌握python面向对象魔术方法（二）</a>
<span class="text-muted">程序员neil</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>接上篇：一文掌握python面向对象魔术方法（一）-CSDN博客目录六、迭代和序列化：1、__iter__(self):定义迭代器，使得类可以被for循环迭代。2、__getitem__(self,key):定义索引操作，如obj[key]。3、__setitem__(self,key,value):定义赋值操作，如obj[key]=value。4、__delitem__(self,key):定义</div>
</li>
<li><a href="/article/1835471185589137408.htm"
title="一文掌握python常用的list（列表）操作" target="_blank">一文掌握python常用的list（列表）操作</a>
<span class="text-muted">程序员neil</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%BC%80%E5%8F%91%E8%AF%AD%E8%A8%80/1.htm">开发语言</a>
<div>目录一、创建列表1.直接创建列表：2.使用list()构造器3.使用列表推导式4.创建空列表二、访问列表元素1.列表支持通过索引访问元素，索引从0开始：2.还可以使用切片操作访问列表的一部分：三、修改列表元素四、添加元素1.append()：在末尾添加元素2.insert()：在指定位置插入元素五、删除元素1.del：删除指定位置的元素2.remove()：删除指定值的第一个匹配项3.pop()：</div>
</li>
<li><a href="/article/1835469798838988800.htm"
title="Python实现简单的机器学习算法" target="_blank">Python实现简单的机器学习算法</a>
<span class="text-muted">master_chenchengg</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E5%8A%9E%E5%85%AC%E6%95%88%E7%8E%87/1.htm">办公效率</a><a class="tag" taget="_blank" href="/search/python%E5%BC%80%E5%8F%91/1.htm">python开发</a><a class="tag" taget="_blank" href="/search/IT/1.htm">IT</a>
<div>Python实现简单的机器学习算法开篇：初探机器学习的奇妙之旅搭建环境：一切从安装开始必备工具箱第一步：安装Anaconda和JupyterNotebook小贴士：如何配置Python环境变量算法初体验：从零开始的Python机器学习线性回归：让数据说话数据准备：从哪里找数据编码实战：Python实现线性回归模型评估：如何判断模型好坏逻辑回归：从分类开始理论入门：什么是逻辑回归代码实现：使用skl</div>
</li>
<li><a href="/article/1835465134710026240.htm"
title="python中的深拷贝与浅拷贝" target="_blank">python中的深拷贝与浅拷贝</a>
<span class="text-muted">anshejd70787</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a>
<div>深拷贝和浅拷贝浅拷贝的时候，修改原来的对象，浅拷贝的对象不会发生改变。1、对象的赋值对象的赋值实际上是对象之间的引用：当创建一个对象，然后将这个对象赋值给另外一个变量的时候，python并没有拷贝这个对象，而只是拷贝了这个对象的引用。当对对象做赋值或者是参数传递或者作为返回值的时候，总是传递原始对象的引用，而不是一个副本。如下所示：>>>aList=["kel","abc",123]>>>bLis</div>
</li>
<li><a href="/article/1835463874560749568.htm"
title="用Python实现简单的猜数字游戏" target="_blank">用Python实现简单的猜数字游戏</a>
<span class="text-muted">程序媛了了</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%B8%B8%E6%88%8F/1.htm">游戏</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>猜数字游戏代码：importrandomdefpythonit():a=random.randint(1,100)n=int(input("输入你猜想的数字："))whilen!=a:ifn>a:print("很遗憾，猜大了")n=int(input("请再次输入你猜想的数字："))elifna::如果玩家猜的数字n大于随机数字a，则输出"很遗憾，猜大了"，并提示玩家再次输入。elifn<a::如</div>
</li>
<li><a href="/article/1835463875160535040.htm"
title="用Python实现读取统计单词个数" target="_blank">用Python实现读取统计单词个数</a>
<span class="text-muted">程序媛了了</span>
<a class="tag" taget="_blank" href="/search/python/1.htm">python</a><a class="tag" taget="_blank" href="/search/%E6%B8%B8%E6%88%8F/1.htm">游戏</a><a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>完整实例代码：fromcollectionsimportCounterdefpythonit():danci={}withopen("pythonit.txt","r",encoding="utf-8")asf:foriinf:words=i.strip().split()forwordinwords:ifwordnotindanci:danci[word]=1else:danci[word]+=</div>
</li>
<li><a href="/article/28.htm"
title="如何用ruby来写hadoop的mapreduce并生成jar包" target="_blank">如何用ruby来写hadoop的mapreduce并生成jar包</a>
<span class="text-muted">wudixiaotie</span>
<a class="tag" taget="_blank" href="/search/mapreduce/1.htm">mapreduce</a>
<div>ruby来写hadoop的mapreduce，我用的方法是rubydoop。怎么配置环境呢：
1.安装rvm：
不说了 网上有
2.安装ruby：
由于我以前是做ruby的，所以习惯性的先安装了ruby，起码调试起来比jruby快多了。
3.安装jruby：
rvm install jruby然后等待安</div>
</li>
<li><a href="/article/155.htm"
title="java编程思想 -- 访问控制权限" target="_blank">java编程思想 -- 访问控制权限</a>
<span class="text-muted">百合不是茶</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E8%AE%BF%E9%97%AE%E6%8E%A7%E5%88%B6%E6%9D%83%E9%99%90/1.htm">访问控制权限</a><a class="tag" taget="_blank" href="/search/%E5%8D%95%E4%BE%8B%E6%A8%A1%E5%BC%8F/1.htm">单例模式</a>
<div>访问权限是java中一个比较中要的知识点,它规定者什么方法可以访问,什么不可以访问
一:包访问权限;
自定义包:
package com.wj.control;
//包
public class Demo {
//定义一个无参的方法
public void DemoPackage(){
System.out.println("调用</div>
</li>
<li><a href="/article/282.htm"
title="[生物与医学]请审慎食用小龙虾" target="_blank">[生物与医学]请审慎食用小龙虾</a>
<span class="text-muted">comsci</span>
<a class="tag" taget="_blank" href="/search/%E7%94%9F%E7%89%A9/1.htm">生物</a>
<div>
现在的餐馆里面出售的小龙虾,有一些是在野外捕捉的,这些小龙虾身体里面可能带有某些病毒和细菌,人食用以后可能会导致一些疾病,严重的甚至会死亡.....
所以,参加聚餐的时候,最好不要点小龙虾...就吃养殖的猪肉,牛肉,羊肉和鱼,等动物蛋白质
</div>
</li>
<li><a href="/article/409.htm"
title="org.apache.jasper.JasperException: Unable to compile class for JSP:" target="_blank">org.apache.jasper.JasperException: Unable to compile class for JSP:</a>
<span class="text-muted">商人shang</span>
<a class="tag" taget="_blank" href="/search/maven/1.htm">maven</a><a class="tag" taget="_blank" href="/search/2.2/1.htm">2.2</a><a class="tag" taget="_blank" href="/search/jdk1.8/1.htm">jdk1.8</a>
<div>环境： jdk1.8 maven tomcat7-maven-plugin 2.0
原因： tomcat7-maven-plugin 2.0 不知吃 jdk 1.8，换成 tomcat7-maven-plugin 2.2就行，即
<plugin>
</div>
</li>
<li><a href="/article/536.htm"
title="你的垃圾你处理掉了吗?GC" target="_blank">你的垃圾你处理掉了吗?GC</a>
<span class="text-muted">oloz</span>
<a class="tag" taget="_blank" href="/search/GC/1.htm">GC</a>
<div>前序:本人菜鸟，此文研究学习来自网络，各位牛牛多指教
1.垃圾收集算法的核心思想
Java语言建立了垃圾收集机制，用以跟踪正在使用的对象和发现并回收不再使用(引用)的对象。该机制可以有效防范动态内存分配中可能发生的两个危险：因内存垃圾过多而引发的内存耗尽，以及不恰当的内存释放所造成的内存非法引用。
垃圾收集算法的核心思想是：对虚拟机可用内存空间，即堆空间中的对象进行识别</div>
</li>
<li><a href="/article/663.htm"
title="shiro 和 SESSSION" target="_blank">shiro 和 SESSSION</a>
<span class="text-muted">杨白白</span>
<a class="tag" taget="_blank" href="/search/shiro/1.htm">shiro</a>
<div>shiro 在web项目里默认使用的是web容器提供的session，也就是说shiro使用的session是web容器产生的，并不是自己产生的，在用于非web环境时可用其他来源代替。在web工程启动的时候它就和容器绑定在了一起，这是通过web.xml里面的shiroFilter实现的。通过session.getSession()方法会在浏览器cokkice产生JESSIONID，当关闭浏览器，此</div>
</li>
<li><a href="/article/790.htm"
title="移动互联网终端 淘宝客如何实现盈利" target="_blank">移动互联网终端 淘宝客如何实现盈利</a>
<span class="text-muted">小桔子</span>
<a class="tag" taget="_blank" href="/search/%E7%A7%BB%E5%8B%95%E5%AE%A2%E6%88%B6%E7%AB%AF/1.htm">移動客戶端</a><a class="tag" taget="_blank" href="/search/%E6%B7%98%E5%AE%A2/1.htm">淘客</a><a class="tag" taget="_blank" href="/search/%E6%B7%98%E5%AF%B6App/1.htm">淘寶App</a>
<div> 2012年淘宝联盟平台为站长和淘宝客带来的分成收入突破30亿元，同比增长100%。而来自移动端的分成达1亿元，其中美丽说、蘑菇街、果库、口袋购物等App运营商分成近5000万元。 可以看出，虽然目前阶段PC端对于淘客而言仍旧是盈利的大头，但移动端已经呈现出爆发之势。而且这个势头将随着智能终端(手机，平板)的加速普及而更加迅猛</div>
</li>
<li><a href="/article/917.htm"
title="wordpress小工具制作" target="_blank">wordpress小工具制作</a>
<span class="text-muted">aichenglong</span>
<a class="tag" taget="_blank" href="/search/wordpress/1.htm">wordpress</a><a class="tag" taget="_blank" href="/search/%E5%B0%8F%E5%B7%A5%E5%85%B7/1.htm">小工具</a>
<div>wordpress 使用侧边栏的小工具，很方便调整页面结构
小工具的制作过程
1 在自己的主题文件中新建一个文件夹(如widget)，在文件夹中创建一个php(AWP_posts-category.php)
小工具是一个类,想侧边栏一样，还得使用代码注册，他才可以再后台使用，基本的代码一层不变
<?php
class AWP_Post_Category extends WP_Wi</div>
</li>
<li><a href="/article/1044.htm"
title="JS微信分享" target="_blank">JS微信分享</a>
<span class="text-muted">AILIKES</span>
<a class="tag" taget="_blank" href="/search/js/1.htm">js</a>
<div>// 所有功能必须包含在 WeixinApi.ready 中进行
WeixinApi.ready(function(Api) {
// 微信分享的数据
var wxData = {
&nb</div>
</li>
<li><a href="/article/1171.htm"
title="封装探讨" target="_blank">封装探讨</a>
<span class="text-muted">百合不是茶</span>
<a class="tag" taget="_blank" href="/search/JAVA%E9%9D%A2%E5%90%91%E5%AF%B9%E8%B1%A1+%E5%B0%81%E8%A3%85/1.htm">JAVA面向对象 封装</a>
<div>//封装 属性 方法 将某些东西包装在一起，通过创建对象或使用静态的方法来调用，称为封装；封装其实就是有选择性地公开或隐藏某些信息，它解决了数据的安全性问题，增加代码的可读性和可维护性
在 Aname类中申明三个属性，将其封装在一个类中：通过对象来调用
例如 1：
//属性 将其设为私有
姓名 name 可以公开
</div>
</li>
<li><a href="/article/1298.htm"
title="jquery radio/checkbox change事件不能触发的问题" target="_blank">jquery radio/checkbox change事件不能触发的问题</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a><a class="tag" taget="_blank" href="/search/jquery/1.htm">jquery</a>
<div>我想让radio来控制当前我选择的是机动车还是特种车，如下所示：
<html>
<head>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js" type="text/javascript"><</div>
</li>
<li><a href="/article/1425.htm"
title="AngularJS中安全性措施" target="_blank">AngularJS中安全性措施</a>
<span class="text-muted">bijian1013</span>
<a class="tag" taget="_blank" href="/search/JavaScript/1.htm">JavaScript</a><a class="tag" taget="_blank" href="/search/AngularJS/1.htm">AngularJS</a><a class="tag" taget="_blank" href="/search/%E5%AE%89%E5%85%A8%E6%80%A7/1.htm">安全性</a><a class="tag" taget="_blank" href="/search/XSRF/1.htm">XSRF</a><a class="tag" taget="_blank" href="/search/JSON%E6%BC%8F%E6%B4%9E/1.htm">JSON漏洞</a>
<div> 在使用web应用中，安全性是应该首要考虑的一个问题。AngularJS提供了一些辅助机制，用来防护来自两个常见攻击方向的网络攻击。
一.JSON漏洞
当使用一个GET请求获取JSON数组信息的时候（尤其是当这一信息非常敏感，</div>
</li>
<li><a href="/article/1552.htm"
title="[Maven学习笔记九]Maven发布web项目" target="_blank">[Maven学习笔记九]Maven发布web项目</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/maven/1.htm">maven</a>
<div>基于Maven的web项目的标准项目结构
user-project
user-core
user-service
user-web
src
</div>
</li>
<li><a href="/article/1679.htm"
title="【Hive七】Hive用户自定义聚合函数(UDAF)" target="_blank">【Hive七】Hive用户自定义聚合函数(UDAF)</a>
<span class="text-muted">bit1129</span>
<a class="tag" taget="_blank" href="/search/hive/1.htm">hive</a>
<div>用户自定义聚合函数，用户提供的多个入参通过聚合计算(求和、求最大值、求最小值)得到一个聚合计算结果的函数。
问题：UDF也可以提供输入多个参数然后输出一个结果的运算，比如加法运算add(3，5)，add这个UDF需要实现UDF的evaluate方法,那么UDF和UDAF的实质分别究竟是什么？
Double evaluate(Double a, Double b)
</div>
</li>
<li><a href="/article/1806.htm"
title="通过 nginx-lua 给 Nginx 增加 OAuth 支持" target="_blank">通过 nginx-lua 给 Nginx 增加 OAuth 支持</a>
<span class="text-muted">ronin47</span>
<div>前言：我们使用Nginx的Lua中间件建立了OAuth2认证和授权层。如果你也有此打算，阅读下面的文档，实现自动化并获得收益。SeatGeek 在过去几年中取得了发展，我们已经积累了不少针对各种任务的不同管理接口。我们通常为新的展示需求创建新模块，比如我们自己的博客、图表等。我们还定期开发内部工具来处理诸如部署、可视化操作及事件处理等事务。在处理这些事务中，我们使用了几个不同的接口来认证：
&n</div>
</li>
<li><a href="/article/1933.htm"
title="利用tomcat-redis-session-manager做session同步时自定义类对象属性保存不上的解决方法" target="_blank">利用tomcat-redis-session-manager做session同步时自定义类对象属性保存不上的解决方法</a>
<span class="text-muted">bsr1983</span>
<a class="tag" taget="_blank" href="/search/session/1.htm">session</a>
<div>在利用tomcat-redis-session-manager做session同步时，遇到了在session保存一个自定义对象时，修改该对象中的某个属性，session未进行序列化，属性没有被存储到redis中。 在 tomcat-redis-session-manager的github上有如下说明： Session Change Tracking
As noted in the &qu</div>
</li>
<li><a href="/article/2060.htm"
title="《代码大全》表驱动法-Table Driven Approach-1" target="_blank">《代码大全》表驱动法-Table Driven Approach-1</a>
<span class="text-muted">bylijinnan</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E7%AE%97%E6%B3%95/1.htm">算法</a>
<div>关于Table Driven Approach的一篇非常好的文章：
http://www.codeproject.com/Articles/42732/Table-driven-Approach
package com.ljn.base;
import java.util.Random;
public class TableDriven {
public </div>
</li>
<li><a href="/article/2187.htm"
title="Sybase封锁原理" target="_blank">Sybase封锁原理</a>
<span class="text-muted">chicony</span>
<a class="tag" taget="_blank" href="/search/Sybase/1.htm">Sybase</a>
<div> 昨天在操作Sybase IQ12.7时意外操作造成了数据库表锁定，不能删除被锁定表数据也不能往其中写入数据。由于着急往该表抽入数据，因此立马着手解决该表的解锁问题。 无奈此前没有接触过Sybase IQ12.7这套数据库产品，加之当时已属于下班时间无法求助于支持人员支持，因此只有借助搜索引擎强大的</div>
</li>
<li><a href="/article/2314.htm"
title="java异常处理机制" target="_blank">java异常处理机制</a>
<span class="text-muted">CrazyMizzz</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a>
<div>java异常关键字有以下几个，分别为 try catch final throw throws
他们的定义分别为
try： Opening exception-handling statement.
catch： Captures the exception.
finally： Runs its code before terminating</div>
</li>
<li><a href="/article/2441.htm"
title="hive 数据插入DML语法汇总" target="_blank">hive 数据插入DML语法汇总</a>
<span class="text-muted">daizj</span>
<a class="tag" taget="_blank" href="/search/hive/1.htm">hive</a><a class="tag" taget="_blank" href="/search/DML/1.htm">DML</a><a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E6%8F%92%E5%85%A5/1.htm">数据插入</a>
<div>Hive的数据插入DML语法汇总1、Loading files into tables语法：1) LOAD DATA [LOCAL] INPATH 'filepath' [OVERWRITE] INTO TABLE tablename [PARTITION (partcol1=val1, partcol2=val2 ...)]解释：1)、上面命令执行环境为hive客户端环境下： hive>l</div>
</li>
<li><a href="/article/2568.htm"
title="工厂设计模式" target="_blank">工厂设计模式</a>
<span class="text-muted">dcj3sjt126com</span>
<a class="tag" taget="_blank" href="/search/%E8%AE%BE%E8%AE%A1%E6%A8%A1%E5%BC%8F/1.htm">设计模式</a>
<div> 使用设计模式是促进最佳实践和良好设计的好办法。设计模式可以提供针对常见的编程问题的灵活的解决方案。 工厂模式
工厂模式（Factory）允许你在代码执行时实例化对象。它之所以被称为工厂模式是因为它负责“生产”对象。工厂方法的参数是你要生成的对象对应的类名称。
Example #1 调用工厂方法（带参数）
<?phpclass Example{ </div>
</li>
<li><a href="/article/2695.htm"
title="mysql字符串查找函数" target="_blank">mysql字符串查找函数</a>
<span class="text-muted">dcj3sjt126com</span>
<a class="tag" taget="_blank" href="/search/mysql/1.htm">mysql</a>
<div>
FIND_IN_SET(str,strlist)
假如字符串str 在由N 子链组成的字符串列表strlist 中，则返回值的范围在1到 N 之间。一个字符串列表就是一个由一些被‘,’符号分开的自链组成的字符串。如果第一个参数是一个常数字符串，而第二个是type SET列，则 FIND_IN_SET() 函数被优化，使用比特计算。如果str不在strlist 或st</div>
</li>
<li><a href="/article/2822.htm"
title="jvm内存管理" target="_blank">jvm内存管理</a>
<span class="text-muted">easterfly</span>
<a class="tag" taget="_blank" href="/search/jvm/1.htm">jvm</a>
<div>一、JVM堆内存的划分
分为年轻代和年老代。年轻代又分为三部分：一个eden,两个survivor。
工作过程是这样的：e区空间满了后，执行minor gc，存活下来的对象放入s0, 对s0仍会进行minor gc，存活下来的的对象放入s1中，对s1同样执行minor gc，依旧存活的对象就放入年老代中；
年老代满了之后会执行major gc，这个是stop the word模式，执行</div>
</li>
<li><a href="/article/2949.htm"
title="CentOS-6.3安装配置JDK-8" target="_blank">CentOS-6.3安装配置JDK-8</a>
<span class="text-muted">gengzg</span>
<a class="tag" taget="_blank" href="/search/centos/1.htm">centos</a>
<div>JAVA_HOME=/usr/java/jdk1.8.0_45
JRE_HOME=/usr/java/jdk1.8.0_45/jre
PATH=$PATH:$JAVA_HOME/bin:$JRE_HOME/bin
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$JRE_HOME/lib
export JAVA_HOME</div>
</li>
<li><a href="/article/3076.htm"
title="【转】关于web路径的获取方法" target="_blank">【转】关于web路径的获取方法</a>
<span class="text-muted">huangyc1210</span>
<a class="tag" taget="_blank" href="/search/Web/1.htm">Web</a><a class="tag" taget="_blank" href="/search/%E8%B7%AF%E5%BE%84/1.htm">路径</a>
<div>假定你的web application 名称为news,你在浏览器中输入请求路径： http://localhost:8080/news/main/list.jsp 则执行下面向行代码后打印出如下结果： 1、 System.out.println(request.getContextPath()); //可返回站点的根路径。也就是项</div>
</li>
<li><a href="/article/3203.htm"
title="php里获取第一个中文首字母并排序" target="_blank">php里获取第一个中文首字母并排序</a>
<span class="text-muted">远去的渡口</span>
<a class="tag" taget="_blank" href="/search/%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84/1.htm">数据结构</a><a class="tag" taget="_blank" href="/search/PHP/1.htm">PHP</a>
<div>很久没来更新博客了，还是觉得工作需要多总结的好。今天来更新一个自己认为比较有成就的问题吧。 最近在做储值结算，需求里结算首页需要按门店的首字母A-Z排序。我的数据结构原本是这样的：
Array
(
[0] => Array
(
[sid] => 2885842
[recetcstoredpay] =&g</div>
</li>
<li><a href="/article/3330.htm"
title="java内部类" target="_blank">java内部类</a>
<span class="text-muted">hm4123660</span>
<a class="tag" taget="_blank" href="/search/java/1.htm">java</a><a class="tag" taget="_blank" href="/search/%E5%86%85%E9%83%A8%E7%B1%BB/1.htm">内部类</a><a class="tag" taget="_blank" href="/search/%E5%8C%BF%E5%90%8D%E5%86%85%E9%83%A8%E7%B1%BB/1.htm">匿名内部类</a><a class="tag" taget="_blank" href="/search/%E6%88%90%E5%91%98%E5%86%85%E9%83%A8%E7%B1%BB/1.htm">成员内部类</a><a class="tag" taget="_blank" href="/search/%E6%96%B9%E6%B3%95%E5%86%85%E9%83%A8%E7%B1%BB/1.htm">方法内部类</a>
<div> 在Java中，可以将一个类定义在另一个类里面或者一个方法里面，这样的类称为内部类。内部类仍然是一个独立的类，在编译之后内部类会被编译成独立的.class文件，但是前面冠以外部类的类名和$符号。内部类可以间接解决多继承问题,可以使用内部类继承一个类，外部类继承一个类，实现多继承。
&nb</div>
</li>
<li><a href="/article/3457.htm"
title="Caused by: java.lang.IncompatibleClassChangeError: class org.hibernate.cfg.Exten" target="_blank">Caused by: java.lang.IncompatibleClassChangeError: class org.hibernate.cfg.Exten</a>
<span class="text-muted">zhb8015</span>
<div>maven pom.xml关于hibernate的配置和异常信息如下，查了好多资料，问题还是没有解决。只知道是包冲突，就是不知道是哪个包....遇到这个问题的分享下是怎么解决的。。
maven pom:
<dependency>
<groupId>org.hibernate</groupId>
<ar</div>
</li>
<li><a href="/article/3584.htm"
title="Spark 性能相关参数配置详解－任务调度篇" target="_blank">Spark 性能相关参数配置详解－任务调度篇</a>
<span class="text-muted">Stark_Summer</span>
<a class="tag" taget="_blank" href="/search/spark/1.htm">spark</a><a class="tag" taget="_blank" href="/search/cache/1.htm">cache</a><a class="tag" taget="_blank" href="/search/cpu/1.htm">cpu</a><a class="tag" taget="_blank" href="/search/%E4%BB%BB%E5%8A%A1%E8%B0%83%E5%BA%A6/1.htm">任务调度</a><a class="tag" taget="_blank" href="/search/yarn/1.htm">yarn</a>
<div>随着Spark的逐渐成熟完善, 越来越多的可配置参数被添加到Spark中来, 本文试图通过阐述这其中部分参数的工作原理和配置思路, 和大家一起探讨一下如何根据实际场合对Spark进行配置优化。
由于篇幅较长，所以在这里分篇组织，如果要看最新完整的网页版内容，可以戳这里：http://spark-config.readthedocs.org/，主要是便</div>
</li>
<li><a href="/article/3711.htm"
title="css3滤镜" target="_blank">css3滤镜</a>
<span class="text-muted">wangkeheng</span>
<a class="tag" taget="_blank" href="/search/html/1.htm">html</a><a class="tag" taget="_blank" href="/search/css/1.htm">css</a>
<div>经常看到一些网站的底部有一些灰色的图标，鼠标移入的时候会变亮，开始以为是js操作src或者bg呢，搜索了一下，发现了一个更好的方法：通过css3的滤镜方法。
html代码：
<a href='' class='icon'><img src='utv.jpg' /></a>
css代码：
.icon{-webkit-filter: graysc</div>
</li>
</ul>
</div>
</div>
</div>
<div>
<div class="container">
<div class="indexes">
<strong>按字母分类：</strong>
<a href="/tags/A/1.htm" target="_blank">A</a><a href="/tags/B/1.htm" target="_blank">B</a><a href="/tags/C/1.htm" target="_blank">C</a><a
href="/tags/D/1.htm" target="_blank">D</a><a href="/tags/E/1.htm" target="_blank">E</a><a href="/tags/F/1.htm" target="_blank">F</a><a
href="/tags/G/1.htm" target="_blank">G</a><a href="/tags/H/1.htm" target="_blank">H</a><a href="/tags/I/1.htm" target="_blank">I</a><a
href="/tags/J/1.htm" target="_blank">J</a><a href="/tags/K/1.htm" target="_blank">K</a><a href="/tags/L/1.htm" target="_blank">L</a><a
href="/tags/M/1.htm" target="_blank">M</a><a href="/tags/N/1.htm" target="_blank">N</a><a href="/tags/O/1.htm" target="_blank">O</a><a
href="/tags/P/1.htm" target="_blank">P</a><a href="/tags/Q/1.htm" target="_blank">Q</a><a href="/tags/R/1.htm" target="_blank">R</a><a
href="/tags/S/1.htm" target="_blank">S</a><a href="/tags/T/1.htm" target="_blank">T</a><a href="/tags/U/1.htm" target="_blank">U</a><a
href="/tags/V/1.htm" target="_blank">V</a><a href="/tags/W/1.htm" target="_blank">W</a><a href="/tags/X/1.htm" target="_blank">X</a><a
href="/tags/Y/1.htm" target="_blank">Y</a><a href="/tags/Z/1.htm" target="_blank">Z</a><a href="/tags/0/1.htm" target="_blank">其他</a>
</div>
</div>
</div>
<footer id="footer" class="mb30 mt30">
<div class="container">
<div class="footBglm">
<a target="_blank" href="/">首页</a> -
<a target="_blank" href="/custom/about.htm">关于我们</a> -
<a target="_blank" href="/search/Java/1.htm">站内搜索</a> -
<a target="_blank" href="/sitemap.txt">Sitemap</a> -
<a target="_blank" href="/custom/delete.htm">侵权投诉</a>
</div>
<div class="copyright">版权所有 IT知识库 CopyRight © 2000-2050 E-COM-NET.COM , All Rights Reserved.
<!-- <a href="https://beian.miit.gov.cn/" rel="nofollow" target="_blank">京ICP备09083238号</a><br>-->
</div>
</div>
</footer>
<!-- 代码高亮 -->
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shCore.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shLegacy.js"></script>
<script type="text/javascript" src="/static/syntaxhighlighter/scripts/shAutoloader.js"></script>
<link type="text/css" rel="stylesheet" href="/static/syntaxhighlighter/styles/shCoreDefault.css"/>
<script type="text/javascript" src="/static/syntaxhighlighter/src/my_start_1.js"></script>
</body>
</html>
