python爬虫中遇到“\xb5”、“xa0”等字符时报错编码错误的处理方式
写python爬虫是遇到编码错误
报错为：
UnicodeEncodeError: 'gbk' codec can't encode character '\xa0'
经过多方查找发现 \xa0是html网页源码中的空格
解决方法
替换掉字符 ：replace(u'\xa0', u' ')
下面是一些html中的常见符号
| chr |
HexCode |
Numeric |
HTML entity |
| " |
\x22 |
" |
" |
| & |
\x26 |
& |
& |
| < |
\x3C |
< |
< |
| > |
\x3E |
> |
> |
| 空格 |
\xA0 |
|
|
| ¡ |
\xA1 |
¡ |
¡ |
| ¢ |
\xA2 |
¢ |
¢ |
| £ |
\xA3 |
£ |
£ |
| ¤ |
\xA4 |
¤ |
¤ |
| ¥ |
\xA5 |
¥ |
¥ |
| ¦ |
\xA6 |
¦ |
¦ |
| § |
\xA7 |
§ |
§ |
| ¨ |
\xA8 |
¨ |
¨ |
| © |
\xA9 |
© |
© |
| ª |
\xAA |
ª |
ª |
| « |
\xAB |
« |
« |
| ¬ |
\xAC |
¬ |
¬ |
|
|
\xAD |
|
|
| ® |
\xAE |
® |
® |
| ¯ |
\xAF |
¯ |
¯ |
| ° |
\xB0 |
° |
° |
| ± |
\xB1 |
± |
± |
| ² |
\xB2 |
² |
² |
| ³ |
\xB3 |
³ |
³ |
| ´ |
\xB4 |
´ |
´ |
| µ |
\xB5 |
µ |
µ |
| ¶ |
\xB6 |
¶ |
¶ |
| · |
\xB7 |
· |
· |
| ¸ |
\xB8 |
¸ |
¸ |
| ¹ |
\xB9 |
¹ |
¹ |
| º |
\xBA |
º |
º |
| » |
\xBB |
» |
» |
| ¼ |
\xBC |
¼ |
¼ |
| ½ |
\xBD |
½ |
½ |
| ¾ |
\xBE |
¾ |
¾ |
| ¿ |
\xBF |
¿ |
¿ |
| × |
\xD7 |
× |
× |
| ÷ |
\xF7 |
÷ |
÷ |
| ƒ |
\u0192 |
ƒ |
ƒ |
| ˆ |
\u02C6 |
ˆ |
ˆ |
| ˜ |
\u02DC |
˜ |
˜ |
|
|
\u2002 |
|
|
|
|
\u2003 |
|
|
|
|
\u2009 |
|
|
|
|
\u200C |
|
|
|
|
\u200D |
|
|
|
|
\u200E |
|
|
|
|
\u200F |
|
|
| – |
\u2013 |
– |
– |
| — |
\u2014 |
— |
— |
| ‘ |
\u2018 |
‘ |
‘ |
| ’ |
\u2019 |
’ |
’ |
| ‚ |
\u201A |
‚ |
‚ |
| “ |
\u201C |
“ |
“ |
| ” |
\u201D |
” |
” |
| „ |
\u201E |
„ |
„ |
| † |
\u2020 |
† |
† |
| ‡ |
\u2021 |
‡ |
‡ |
| • |
\u2022 |
• |
• |
| … |
\u2026 |
… |
… |
| ‰ |
\u2030 |
‰ |
‰ |
| ′ |
\u2032 |
′ |
′ |
| ″ |
\u2033 |
″ |
″ |
| ‹ |
\u2039 |
‹ |
‹ |
| › |
\u203A |
› |
› |
| ‾ |
\u203E |
‾ |
‾ |
| ⁄ |
\u2044 |
⁄ |
⁄ |
| € |
\u20AC |
€ |
€ |
| ℑ |
\u2111 |
ℑ |
ℑ |
| ? |
\u2113 |
ℓ |
|
| № |
\u2116 |
№ |
|
| ℘ |
\u2118 |
℘ |
℘ |
| ℜ |
\u211C |
ℜ |
ℜ |
| ™ |
\u2122 |
™ |
™ |
| ℵ |
\u2135 |
ℵ |
ℵ |
| ← |
\u2190 |
← |
← |
| ↑ |
\u2191 |
↑ |
↑ |
| → |
\u2192 |
→ |
→ |
| ↓ |
\u2193 |
↓ |
↓ |
| ↔ |
\u2194 |
↔ |
↔ |
| ↵ |
\u21B5 |
↵ |
↵ |
| ⇐ |
\u21D0 |
⇐ |
⇐ |
| ⇑ |
\u21D1 |
⇑ |
⇑ |
| ⇒ |
\u21D2 |
⇒ |
⇒ |
| ⇓ |
\u21D3 |
⇓ |
⇓ |
| ⇔ |
\u21D4 |
⇔ |
⇔ |
| ∀ |
\u2200 |
∀ |
∀ |
| ∂ |
\u2202 |
∂ |
∂ |
| ∃ |
\u2203 |
∃ |
∃ |
| ∅ |
\u2205 |
∅ |
∅ |
| ∇ |
\u2207 |
∇ |
∇ |
| ∈ |
\u2208 |
∈ |
∈ |
| ∉ |
\u2209 |
∉ |
∉ |
| ∋ |
\u220B |
∋ |
∋ |
| ∏ |
\u220F |
∏ |
∏ |
| ∑ |
\u2211 |
∑ |
∑ |
| − |
\u2212 |
− |
− |
| ∗ |
\u2217 |
∗ |
∗ |
| √ |
\u221A |
√ |
√ |
| ∝ |
\u221D |
∝ |
∝ |
| ∞ |
\u221E |
∞ |
∞ |
| ∠ |
\u2220 |
∠ |
∠ |
| ∧ |
\u2227 |
∧ |
∧ |
| ∨ |
\u2228 |
∨ |
∨ |
| ∩ |
\u2229 |
∩ |
∩ |
| ∪ |
\u222A |
∪ |
∪ |
| ∫ |
\u222B |
∫ |
∫ |
| ∴ |
\u2234 |
∴ |
∴ |
| ∼ |
\u223C |
∼ |
∼ |
| ≅ |
\u2245 |
≅ |
≅ |
| ≈ |
\u2248 |
≈ |
≈ |
| ≠ |
\u2260 |
≠ |
≠ |
| ≡ |
\u2261 |
≡ |
≡ |
| ≤ |
\u2264 |
≤ |
≤ |
| ≥ |
\u2265 |
≥ |
≥ |
| ⊂ |
\u2282 |
⊂ |
⊂ |
| ⊃ |
\u2283 |
⊃ |
⊃ |
| ⊄ |
\u2284 |
⊄ |
⊄ |
| ⊆ |
\u2286 |
⊆ |
⊆ |
| ⊇ |
\u2287 |
⊇ |
⊇ |
| ⊕ |
\u2295 |
⊕ |
⊕ |
| ⊗ |
\u2297 |
⊗ |
⊗ |
| ⊥ |
\u22A5 |
⊥ |
⊥ |
| ⋅ |
\u22C5 |
⋅ |
⋅ |
| ⌈ |
\u2308 |
⌈ |
⌈ |
| ⌉ |
\u2309 |
⌉ |
⌉ |
| ⌊ |
\u230A |
⌊ |
⌊ |
| ⌋ |
\u230B |
⌋ |
⌋ |
| 〈 |
\u2329 |
〈 |
⟨ |
| 〉 |
\u232A |
〉 |
⟩ |
| ◊ |
\u25CA |
◊ |
◊ |
| ♠ |
\u2660 |
♠ |
♠ |
| ♣ |
\u2663 |
♣ |
♣ |
| ♥ |
\u2665 |
♥ |
♥ |
| ♦ |
\u2666 |
♦ |
♦ |