作为非文艺青年,非技术青年的同学,闲暇(或忙碌但不想做事情的时候)浏览下豆瓣的日志总是种乐趣。发现豆瓣上的好多人很有趣,比如画彩铅的同学,有这样的兴趣真的很幸福。
闲话少说。这里是用python程序统计的豆瓣中喜欢数最多的400篇日志,觉得其中很多比较有趣的内容,所以与大家分享。
注:这是个很简单的抓取程序,准确率会随着抓取时间提高,目前的这个列表肯定非常不完善,而且日志的喜欢数肯定是不断变化的,所以这只是部分比较流行的日志。
豆瓣日志链接见http://www.douban.com/note/240364805/
2012-10-07 09:31:11
日志链接,被喜欢数
http://www.douban.com/note/153041669/
15586
http://www.douban.com/note/202071868/
10994
http://www.douban.com/note/211252745/
10976
http://www.douban.com/note/207404332/
9989
http://www.douban.com/note/205670967/
9642
http://www.douban.com/note/96100372/
8410
http://www.douban.com/note/175390726/
8376
http://www.douban.com/note/207485385/
7630
http://www.douban.com/note/216068921/
7420
http://www.douban.com/note/220790644/
7292
http://www.douban.com/note/219499758/
7176
http://www.douban.com/note/135569482/
7126
http://www.douban.com/note/138959466/
7119
http://www.douban.com/note/238241656/
7107
http://www.douban.com/note/212633927/
6421
http://www.douban.com/note/210814135/
6367
http://www.douban.com/note/226587965/
6342
http://www.douban.com/note/72019296/
6173
http://www.douban.com/note/151013172/
5857
http://www.douban.com/note/207368638/
5842
http://www.douban.com/note/168683372/
5617
http://www.douban.com/note/225532822/
5293
http://www.douban.com/note/239064780/
5145
http://www.douban.com/note/238136093/
5007
http://www.douban.com/note/224894229/
4915
http://www.douban.com/note/204635366/
4857
http://www.douban.com/note/220441371/
4841
http://www.douban.com/note/224144181/
4773
http://www.douban.com/note/83591249/
4615
http://www.douban.com/note/225344369/
4590
http://www.douban.com/note/193930649/
4579
http://www.douban.com/note/203140756/
4556
http://www.douban.com/note/214412846/
4556
http://www.douban.com/note/208197789/
4500
http://www.douban.com/note/200155071/
4354
http://www.douban.com/note/212306651/
4298
http://www.douban.com/note/219769009/
4298
http://www.douban.com/note/63758986/
4294
http://www.douban.com/note/223090457/
4265
http://www.douban.com/note/160038219/
4256
http://www.douban.com/note/228798617/
4129
http://www.douban.com/note/152452345/
4125
http://www.douban.com/note/207547341/
4100
http://www.douban.com/note/220925751/
3994
http://www.douban.com/note/224488780/
3953
http://www.douban.com/note/192984823/
3901
http://www.douban.com/note/232472509/
3656
http://www.douban.com/note/205492628/
3598
http://www.douban.com/note/231582385/
3597
http://www.douban.com/note/233000010/
3573
http://www.douban.com/note/200614238/
3500
http://www.douban.com/note/232372907/
3471
http://www.douban.com/note/230570041/
3443
http://www.douban.com/note/223397197/
3367
http://www.douban.com/note/235202251/
3223
http://www.douban.com/note/226173156/
3205
http://www.douban.com/note/68185576/
3189
http://www.douban.com/note/196461710/
3148
http://www.douban.com/note/230726178/
3140
http://www.douban.com/note/236759556/
3132
http://www.douban.com/note/223118853/
3131
http://www.douban.com/note/238771851/
3128
http://www.douban.com/note/237519821/
3081
http://www.douban.com/note/215948214/
3066
http://www.douban.com/note/226916825/
2990
http://www.douban.com/note/220226024/
2957
http://www.douban.com/note/228750983/
2936
http://www.douban.com/note/226289388/
2929
http://www.douban.com/note/149806395/
2869
http://www.douban.com/note/228499395/
2846
http://www.douban.com/note/219495038/
2786
http://www.douban.com/note/234216806/
2762
http://www.douban.com/note/212918016/
2657
http://www.douban.com/note/237695424/
2646
http://www.douban.com/note/210952785/
2640
http://www.douban.com/note/193780200/
2632
http://www.douban.com/note/204338310/
2622
http://www.douban.com/note/208919273/
2620
http://www.douban.com/note/234284919/
2558
http://www.douban.com/note/209640960/
2540
http://www.douban.com/note/237023266/
2529
http://www.douban.com/note/237265588/
2486
http://www.douban.com/note/189392337/
2472
http://www.douban.com/note/221962564/
2466
http://www.douban.com/note/165902233/
2448
http://www.douban.com/note/237342548/
2433
http://www.douban.com/note/226523297/
2426
http://www.douban.com/note/236126626/
2396
http://www.douban.com/note/226531501/
2377
http://www.douban.com/note/238524198/
2369
http://www.douban.com/note/224792869/
2338
http://www.douban.com/note/235760849/
2336
http://www.douban.com/note/226446968/
2329
http://www.douban.com/note/238372921/
2328
http://www.douban.com/note/194516836/
2327
http://www.douban.com/note/228933837/
2290
http://www.douban.com/note/239360056/
2287
http://www.douban.com/note/229761323/
2276
http://www.douban.com/note/214867700/
2276
http://www.douban.com/note/226898175/
2271
http://www.douban.com/note/237517252/
2263
http://www.douban.com/note/231986759/
2232
http://www.douban.com/note/234972679/
2193
http://www.douban.com/note/124392784/
2141
http://www.douban.com/note/227016058/
2082
http://www.douban.com/note/238538671/
2058
http://www.douban.com/note/182405697/
2051
http://www.douban.com/note/238221410/
2049
http://www.douban.com/note/237876678/
2028
http://www.douban.com/note/236818988/
2021
http://www.douban.com/note/231003533/
2019
http://www.douban.com/note/230258884/
2012
http://www.douban.com/note/214549277/
1983
http://www.douban.com/note/235563083/
1975
http://www.douban.com/note/212449992/
1958
http://www.douban.com/note/236167841/
1956
http://www.douban.com/note/227405774/
1941
http://www.douban.com/note/236056766/
1908
http://www.douban.com/note/216488639/
1906
http://www.douban.com/note/209371890/
1896
http://www.douban.com/note/233822258/
1892
http://www.douban.com/note/236418064/
1878
http://www.douban.com/note/167797624/
1871
http://www.douban.com/note/156046788/
1865
http://www.douban.com/note/215368964/
1852
http://www.douban.com/note/212544450/
1846
http://www.douban.com/note/204852081/
1826
http://www.douban.com/note/230160390/
1825
http://www.douban.com/note/237366885/
1815
http://www.douban.com/note/226354979/
1811
http://www.douban.com/note/66496487/
1795
http://www.douban.com/note/234822974/
1786
http://www.douban.com/note/218535116/
1770
http://www.douban.com/note/217444792/
1763
http://www.douban.com/note/230059284/
1757
http://www.douban.com/note/235924606/
1757
http://www.douban.com/note/237252160/
1749
http://www.douban.com/note/167068151/
1736
http://www.douban.com/note/232519056/
1724
http://www.douban.com/note/212506400/
1721
http://www.douban.com/note/239097115/
1691
http://www.douban.com/note/229569828/
1689
http://www.douban.com/note/233034351/
1681
http://www.douban.com/note/212167109/
1666
http://www.douban.com/note/228670320/
1662
http://www.douban.com/note/148007212/
1657
http://www.douban.com/note/234184135/
1645
http://www.douban.com/note/234782629/
1615
http://www.douban.com/note/235241271/
1598
http://www.douban.com/note/80046555/
1597
http://www.douban.com/note/239969308/
1556
http://www.douban.com/note/230959741/
1553
http://www.douban.com/note/234902167/
1549
http://www.douban.com/note/234825907/
1539
http://www.douban.com/note/235793634/
1535
http://www.douban.com/note/211197993/
1533
http://www.douban.com/note/236053490/
1521
http://www.douban.com/note/232194746/
1520
http://www.douban.com/note/235306912/
1513
http://www.douban.com/note/230512084/
1512
http://www.douban.com/note/197840922/
1511
http://www.douban.com/note/237274733/
1499
http://www.douban.com/note/222925776/
1472
http://www.douban.com/note/175402804/
1464
http://www.douban.com/note/229189614/
1457
http://www.douban.com/note/235553432/
1456
http://www.douban.com/note/226781192/
1447
http://www.douban.com/note/123734431/
1446
http://www.douban.com/note/210748528/
1442
http://www.douban.com/note/157070932/
1440
http://www.douban.com/note/94116348/
1429
http://www.douban.com/note/235608394/
1423
http://www.douban.com/note/220183302/
1404
http://www.douban.com/note/197478617/
1389
http://www.douban.com/note/220438288/
1380
http://www.douban.com/note/209812966/
1369
http://www.douban.com/note/234420349/
1359
http://www.douban.com/note/225368696/
1351
http://www.douban.com/note/170148059/
1348
http://www.douban.com/note/188777419/
1337
http://www.douban.com/note/207676029/
1318
http://www.douban.com/note/214462677/
1296
http://www.douban.com/note/208213495/
1280
http://www.douban.com/note/221426825/
1268
http://www.douban.com/note/212652158/
1250
http://www.douban.com/note/200349519/
1246
http://www.douban.com/note/218459562/
1236
http://www.douban.com/note/94646533/
1231
http://www.douban.com/note/233626011/
1230
http://www.douban.com/note/240088013/
1230
http://www.douban.com/note/237921745/
1226
http://www.douban.com/note/190003206/
1212
http://www.douban.com/note/212901271/
1206
http://www.douban.com/note/231467295/
1191
http://www.douban.com/note/224880958/
1181
http://www.douban.com/note/221351137/
1174
http://www.douban.com/note/225028938/
1172
http://www.douban.com/note/204606093/
1159
http://www.douban.com/note/76613787/
1159
http://www.douban.com/note/226531667/
1157
http://www.douban.com/note/224985454/
1154
http://www.douban.com/note/235798683/
1152
http://www.douban.com/note/239761554/
1144
http://www.douban.com/note/233033797/
1142
http://www.douban.com/note/235214961/
1138
http://www.douban.com/note/201556032/
1130
http://www.douban.com/note/239752450/
1114
http://www.douban.com/note/233121523/
1105
http://www.douban.com/note/239652100/
1098
http://www.douban.com/note/139881623/
1096
http://www.douban.com/note/78105058/
1075
http://www.douban.com/note/197928413/
1071
http://www.douban.com/note/206910340/
1070
http://www.douban.com/note/225178197/
1065
http://www.douban.com/note/231563428/
1056
http://www.douban.com/note/239659426/
1052
http://www.douban.com/note/201935064/
1051
http://www.douban.com/note/234214384/
1045
http://www.douban.com/note/217441818/
1035
http://www.douban.com/note/214237336/
1031
http://www.douban.com/note/228027251/
1015
http://www.douban.com/note/206713392/
1014
http://www.douban.com/note/228842046/
999
http://www.douban.com/note/230276217/
994
http://www.douban.com/note/228688045/
992
http://www.douban.com/note/230985403/
986
http://www.douban.com/note/215882959/
978
http://www.douban.com/note/188113226/
965
http://www.douban.com/note/235078668/
964
http://www.douban.com/note/225544573/
961
http://www.douban.com/note/62935279/
958
http://www.douban.com/note/239639966/
952
http://www.douban.com/note/218985019/
947
http://www.douban.com/note/128688737/
945
http://www.douban.com/note/227491851/
936
http://www.douban.com/note/230823805/
935
http://www.douban.com/note/194538770/
934
http://www.douban.com/note/232191793/
934
http://www.douban.com/note/233998222/
931
http://www.douban.com/note/150387644/
928
http://www.douban.com/note/231312249/
919
http://www.douban.com/note/206040896/
911
http://www.douban.com/note/206850188/
895
http://www.douban.com/note/233552929/
892
http://www.douban.com/note/229290007/
880
http://www.douban.com/note/226691869/
861
http://www.douban.com/note/161442014/
857
http://www.douban.com/note/161307302/
853
http://www.douban.com/note/235139719/
851
http://www.douban.com/note/88789287/
850
http://www.douban.com/note/236499319/
849
http://www.douban.com/note/235540249/
845
http://www.douban.com/note/220171367/
845
http://www.douban.com/note/238566749/
843
http://www.douban.com/note/228311097/
841
http://www.douban.com/note/237975829/
834
http://www.douban.com/note/236955723/
832
http://www.douban.com/note/236357825/
830
http://www.douban.com/note/228745252/
830
http://www.douban.com/note/208377659/
820
http://www.douban.com/note/215626929/
818
http://www.douban.com/note/220433175/
808
http://www.douban.com/note/238161802/
802
http://www.douban.com/note/80506357/
800
http://www.douban.com/note/237926673/
795
http://www.douban.com/note/149985745/
793
http://www.douban.com/note/224863226/
783
http://www.douban.com/note/234689839/
781
http://www.douban.com/note/231292771/
780
http://www.douban.com/note/227487804/
778
http://www.douban.com/note/239100962/
773
http://www.douban.com/note/210828706/
769
http://www.douban.com/note/228664838/
759
http://www.douban.com/note/200551727/
756
http://www.douban.com/note/224815722/
753
http://www.douban.com/note/188955894/
743
http://www.douban.com/note/237391702/
741
http://www.douban.com/note/232457886/
740
http://www.douban.com/note/156080259/
740
http://www.douban.com/note/174726455/
739
http://www.douban.com/note/218732338/
733
http://www.douban.com/note/233006133/
730
http://www.douban.com/note/236095244/
729
http://www.douban.com/note/237302819/
727
http://www.douban.com/note/202392155/
726
http://www.douban.com/note/228675371/
725
http://www.douban.com/note/205332158/
719
http://www.douban.com/note/220788918/
718
http://www.douban.com/note/187356721/
718
http://www.douban.com/note/228510065/
717
http://www.douban.com/note/137565463/
714
http://www.douban.com/note/239125372/
712
http://www.douban.com/note/221658514/
711
http://www.douban.com/note/167345314/
709
http://www.douban.com/note/208179913/
709
http://www.douban.com/note/238197766/
704
http://www.douban.com/note/197871745/
702
http://www.douban.com/note/201572527/
699
http://www.douban.com/note/238794858/
694
http://www.douban.com/note/235226548/
693
http://www.douban.com/note/220716569/
693
http://www.douban.com/note/211686832/
689
http://www.douban.com/note/227410940/
679
http://www.douban.com/note/236738028/
672
http://www.douban.com/note/201180910/
669
http://www.douban.com/note/234438150/
665
http://www.douban.com/note/96703670/
665
http://www.douban.com/note/239090787/
645
http://www.douban.com/note/237832422/
640
http://www.douban.com/note/149471238/
636
http://www.douban.com/note/171496171/
632
http://www.douban.com/note/161471673/
630
http://www.douban.com/note/206271662/
630
http://www.douban.com/note/223856907/
626
http://www.douban.com/note/226886470/
624
http://www.douban.com/note/214653847/
624
http://www.douban.com/note/205088865/
607
http://www.douban.com/note/238072346/
604
http://www.douban.com/note/235233767/
601
http://www.douban.com/note/235516345/
601
http://www.douban.com/note/220623542/
598
http://www.douban.com/note/236491436/
597
http://www.douban.com/note/231088165/
595
http://www.douban.com/note/160793626/
582
http://www.douban.com/note/194472965/
570
http://www.douban.com/note/222068059/
569
http://www.douban.com/note/213634767/
568
http://www.douban.com/note/237005846/
559
http://www.douban.com/note/92431123/
559
http://www.douban.com/note/191222912/
557
http://www.douban.com/note/137234963/
553
http://www.douban.com/note/205802564/
543
http://www.douban.com/note/220718726/
542
http://www.douban.com/note/224846875/
541
http://www.douban.com/note/206975500/
539
http://www.douban.com/note/149786780/
536
http://www.douban.com/note/198177784/
535
http://www.douban.com/note/120924182/
533
http://www.douban.com/note/238187347/
532
http://www.douban.com/note/203980826/
530
http://www.douban.com/note/195161245/
520
http://www.douban.com/note/130759266/
509
http://www.douban.com/note/233399682/
508
http://www.douban.com/note/168495419/
508
http://www.douban.com/note/237343861/
506
http://www.douban.com/note/207465887/
504
http://www.douban.com/note/205932033/
503
http://www.douban.com/note/148625029/
502
http://www.douban.com/note/221540111/
500
http://www.douban.com/note/211249667/
495
http://www.douban.com/note/232566491/
494
http://www.douban.com/note/228792096/
492
http://www.douban.com/note/237287392/
483
http://www.douban.com/note/186517481/
481
http://www.douban.com/note/228422811/
477
http://www.douban.com/note/236540265/
477
http://www.douban.com/note/202575876/
475
http://www.douban.com/note/197968123/
474
http://www.douban.com/note/224706279/
473
http://www.douban.com/note/202807390/
470
http://www.douban.com/note/205933021/
470
http://www.douban.com/note/221089050/
467
http://www.douban.com/note/180252351/
467
http://www.douban.com/note/159363290/
467
http://www.douban.com/note/231940783/
466
http://www.douban.com/note/222440941/
465
http://www.douban.com/note/233144061/
464
http://www.douban.com/note/237276071/
460
http://www.douban.com/note/228177271/
458
http://www.douban.com/note/149061698/
454
http://www.douban.com/note/213407012/
452
http://www.douban.com/note/238085085/
450
http://www.douban.com/note/212743268/
449
http://www.douban.com/note/51412995/
444
http://www.douban.com/note/233507664/
441
http://www.douban.com/note/221274070/
440
http://www.douban.com/note/239213599/
438
http://www.douban.com/note/230566022/
433
http://www.douban.com/note/119216007/
433
http://www.douban.com/note/238420502/
433
http://www.douban.com/note/232775250/
432
http://www.douban.com/note/236149011/
431
http://www.douban.com/note/75838426/
430
http://www.douban.com/note/225653735/
430
http://www.douban.com/note/223033633/
427
http://www.douban.com/note/232904418/
426
http://www.douban.com/note/224154164/
425
http://www.douban.com/note/218996804/
423
http://www.douban.com/note/228434344/
423
http://www.douban.com/note/227950844/
420
http://www.douban.com/note/155349803/
418
http://www.douban.com/note/217131130/
416
http://www.douban.com/note/233518875/
412
http://www.douban.com/note/208818005/
412
http://www.douban.com/note/184241030/
411
http://www.douban.com/note/207296075/
407
http://www.douban.com/note/163226788/
406
http://www.douban.com/note/211526076/
405
http://www.douban.com/note/207231362/
405
http://www.douban.com/note/232554555/ 402
Python程序抓取
用法: python douban.py > log.txt
douban.py如下:
import urllib2
import httplib2
from BeautifulSoup import BeautifulSoup, SoupStrainer
import re
#import Queue
import heapq
import time
from time import strftime
from urllib import FancyURLopener
from random import choice
user_agents = [
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4',
'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11',
'Opera/9.25 (Windows NT 5.1; U; en)',
'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)',
'Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Kubuntu)',
'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070731 Ubuntu/dapper-security Firefox/1.5.0.12',
'Lynx/2.8.5rel.1 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/1.2.9'
]
headers = {
'Accept':'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Charset':'GBK,utf-8;q=0.7,*;q=0.3',
'Accept-Encoding':'gzip,deflate,sdch',
'Accept-Language':'zh-CN,zh;q=0.8,en;q=0.6',
'Cache-Control':'max-age=0',
'Connection':'keep-alive',
'Cookie':'bid="cRcN3O9ZJHU"; dbcl2="2359624:xkxvOaiLLTU"; ck="XxRv"; ll="108288"',
'Host':'www.douban.com',
'User-Agent':'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.79 Safari/537.4'
}
class MyOpener(FancyURLopener, object):
version = choice(user_agents)
#myopener = MyOpener()
#print myopener.open('http://www.douban.com/').read()
class PageWithLikes():
def __init__(self, url, numlikes):
self.url = url
self.numlikes = numlikes
def __cmp__(self, other):
return cmp(other.numlikes, self.numlikes)
def dumppopular(populars, qsize):
#print "[dumppopular]before: len(populars)", len(populars)
populars.sort()
with open("populars" + str(qsize) + ".txt", "w") as f:
f.write(strftime("%Y-%m-%d %H:%M:%S", time.localtime()) + "\n")
for popular in populars:
f.write(popular.url + "\t" + str(popular.numlikes) + "\n")
#print "[dumppopular]after: len(populars)", len(populars)
#print "qsize: ", populars.qsize()
def getpopular(populars, url):
#for url in "http://www.douban.com/note/239090787/", "http://www.douban.com/note/239349810/":
#url = "http://www.douban.com/note/239090787/"
try:
time.sleep(3)
#page = urllib2.urlopen(url)
#soup = BeautifulSoup(page)
print url
visitedurls.add(url[:44])
status, response = http.request(url, headers=headers)
#print status
soup = BeautifulSoup(response, parseOnlyThese=SoupStrainer('span'))
likespan = soup.findAll('span', {"class": "fav-num"})[0]
numlikes = int(re.search('\d+', likespan.contents[1].contents[0]).group())
if int(numlikes) > 400:
#print url
#populars.put(PageWithLikes(url, numlikes))
heapq.heappush(populars, PageWithLikes(url, numlikes))
qsize = len(populars)
if qsize % 100 == 0:
print "qsize: ", qsize
dumppopular(populars, qsize)
except Exception, e:
print e
pass
if __name__ == "__main__":
url = "http://www.douban.com"
#noteurls = list()
visitedurls = set()
tovisiturls = [url]
http = httplib2.Http()
notepattern = re.compile("^http://www.douban.com/note/\d+/?$")
#populars = Queue.PriorityQueue(maxsize = 1000)
populars = list()
print strftime("%Y-%m-%d %H:%M:%S", time.localtime())
while len(tovisiturls) > 0 and len(populars) < 1000:
url = tovisiturls.pop()
visitedurls.add(url[:44])
print url
try:
status, response = http.request(url, headers=headers)
time.sleep(3)
#print status
#print response
for link in BeautifulSoup(response, parseOnlyThese=SoupStrainer('a')):
if link.has_key('href'):
href = link['href']
if href.startswith("http://www.douban.com") and not href.startswith("http://www.douban.com/feed") and href[:44] not in visitedurls:
#print href
if re.match(notepattern, href):
tovisiturls.append(href)
#noteurls.append(href)
getpopular(populars, href)
elif href.endswith("notes/") or href.endswith("notes"):
tovisiturls.append(href)
else:
tovisiturls.insert(0, href)
except Exception, e:
print e
pass