一 前言
1.1 60w知乎网名的数据从何而来？
去年在接触Java爬虫的时候，接触到了一个关于知乎的爬虫。个人觉得写的非常好，当时抓取的效率和成功率还是特别特别高，现在可能知乎反扒做的更好，这个开源知乎爬虫没之前抓取的那么顺利了。我记得当时在我的i7+8g的机器上爬了将近两天，大概爬取了60多w的数据。当然，实际抓取的用户数据数量肯定比这个多，只是持久化过程不同步而已，也就是抓取的好几个用户可能只有一个存入数据库中。
最后，本文提供的知乎网名数据是2017年12月份左右抓取的数据。
60w知乎网名数据：
链接：https://pan.baidu.com/s/1pEcHAbRyKP7KGcgTuZfPVA 密码：l3so
项目源码地址(如果觉得 有帮助的话，欢迎给个Star.)：
https://github.com/Snailclimb/J2ee-Advanced （Java Web进阶学习的一些源码加详细讲解）
1.2 通过本篇文章你能学到什么？
- SSM环境的搭建；
- 如何在SSM项目中使用Echarts
1.3 效果图展示
细心的同学会发现，我其实只从数据库抓取了9条数据出来。因为我的SQL语句写错了（逃....），大家可以自己修改Mapper文件。
二 SSM环境搭建
声明一下，笔主使用的是MyEclipse2016(主要是为了暑假做的项目的编码环境的统一，所以我选择了MyEclipse2016)。
2.1 项目结构
2.2 配置文件
2.3.1 pom.xml
需要的jar包，都在这里配置好。另外我配置了一个Tomcat插件，这样就可以通过Maven Build的方式来运行项目了。具体运行方式如下：
右键项目->run as -> Maven build
然后输入tomcat7:run后点击run即可
这里提一点：@ResponseBody注解要把对象转换成json格式，所以需要添加相关转换依赖的jar包（jackson）
pom.xml
4.0.0
spring
ssm-echarts-demo
war
0.0.1-SNAPSHOT
ssm-echarts-demo Maven Webapp
http://maven.apache.org
3.0
UTF-8
4.0.0.RELEASE
3.1.1
1.7.4
1.4
1.6
2.8.9
1.1.1
5.1.29
1.2.1
1.3.1
2.4
2.0.0.Final
5.4.1.Final
junit
junit
4.12
test
org.springframework
spring-test
4.0.0.RELEASE
test
commons-logging
commons-logging
1.1.3
commons-collections
commons-collections
3.2.1
javax.servlet
jstl
1.2
org.mybatis
mybatis
${mybatis.version}
org.mybatis
mybatis-spring
${mybatis-spring.version}
mysql
mysql-connector-java
${mysql.version}
com.alibaba
druid
0.2.23
org.springframework
spring-core
${spring.version}
org.aspectj
aspectjrt
${aspectJ.version}
org.aspectj
aspectjweaver
${aspectJ.version}
org.springframework
spring-context
${spring.version}
org.springframework
spring-aop
${spring.version}
org.springframework
spring-tx
${spring.version}
org.springframework
spring-jdbc
${spring.version}
org.springframework
spring-web
${spring.version}
org.springframework
spring-webmvc
${spring.version}
org.springframework
spring-context-support
${spring.version}
com.google.code.gson
gson
2.8.5
org.codehaus.jackson
jackson-mapper-asl
1.9.13
com.alibaba
fastjson
1.2.3
com.fasterxml.jackson.core
jackson-annotations
${jackson.version}
com.fasterxml.jackson.core
jackson-core
${jackson.version}
com.fasterxml.jackson.core
jackson-databind
${jackson.version}
javax.servlet
javax.servlet-api
3.1.0
provided
commons-dbcp
commons-dbcp
${commons-dbcp.version}
commons-pool
commons-pool
${commons-pool.version}
commons-fileupload
commons-fileupload
${commons-fileupload.version}
commons-io
commons-io
${commons-io.version}
javax.validation
validation-api
${validation-api.version}
org.hibernate
hibernate-validator
${hibernate-validator.version}
ssm-echarts-demo
maven-compiler-plugin
2.3.2
1.8
1.8
org.apache.tomcat.maven
tomcat7-maven-plugin
/
8082
maven-war-plugin
2.6
false
src/main/resources
**/*.properties
**/*.xml
**/*.tld
false
src/main/java
**/*.properties
**/*.xml
**/*.tld
false
2.3.2 Spring相关配置文件：
由于Spring配置较多所以我这里分了4层，分别是dao、service、transaction、web。
applicationContext-dao.xml
applicationContext-service.xml
applicationContext-trans.xml:
springmvc.xml
10485760
UTF-8
classpath:validateMessage
2.3.3 数据库连接以及log4j配置文件：
Log4J的配置文件(Configuration File)就是用来设置记录器的级别、存放器和布局的，它可接key=value格式的设置或xml格式的设置信息。通过配置，可以创建出Log4J的运行环境。
log4j.properties
log4j.rootLogger=DEBUG, stdout
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d [%-5p] %c - %m%ndb.properties
jdbc.driverClassName=com.mysql.jdbc.Driver
jdbc.url=jdbc:mysql://localhost:3306/spider_zhihu_crawler
jdbc.username=root
jdbc.password=xxxxxx2.3.4 Mybatis配置文件：
spring-mybatis.xml
2.3.5 web.xml：
web.xml
ssm-echarts-demo
index.html
index.htm
index.jsp
default.html
default.htm
default.jsp
default
org.apache.catalina.servlets.DefaultServlet
debug
0
listings
false
1
contextConfigLocation
classpath:spring/applicationContext-*.xml
org.springframework.web.context.ContextLoaderListener
default
*.js
default
*.css
default
*.gif
default
*.jpg
default
*.ico
default
*.png
default
*.htm
default
*.eot
default
*.svg
default
*.ttf
default
*.woff
default
*.json
spring
org.springframework.web.servlet.DispatcherServlet
contextConfigLocation
classpath:spring/springmvc.xml
1
spring
/
Encoding
org.springframework.web.filter.CharacterEncodingFilter
encoding
UTF-8
Encoding
*
三 核心代码
3.1 bean
User.java
package com.snailclimb.bean;
/**
* 用户实体类
*
* @author Snailclimb
*
*/
public class User {
private Integer id;
private String userToken;
private String location;
private String business;
private String sex;
private String employment;
private String education;
private String username;
private String url;
private Integer agrees;
private Integer thanks;
private Integer asks;
private Integer answers;
private Integer posts;
private Integer followees;
private Integer followers;
private String hashid;
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
public String getUserToken() {
return userToken;
}
public void setUserToken(String userToken) {
this.userToken = userToken == null ? null : userToken.trim();
}
public String getLocation() {
return location;
}
public void setLocation(String location) {
this.location = location == null ? null : location.trim();
}
public String getBusiness() {
return business;
}
public void setBusiness(String business) {
this.business = business == null ? null : business.trim();
}
public String getSex() {
return sex;
}
public void setSex(String sex) {
this.sex = sex == null ? null : sex.trim();
}
public String getEmployment() {
return employment;
}
public void setEmployment(String employment) {
this.employment = employment == null ? null : employment.trim();
}
public String getEducation() {
return education;
}
public void setEducation(String education) {
this.education = education == null ? null : education.trim();
}
public String getUsername() {
return username;
}
public void setUsername(String username) {
this.username = username == null ? null : username.trim();
}
public String getUrl() {
return url;
}
public void setUrl(String url) {
this.url = url == null ? null : url.trim();
}
public Integer getAgrees() {
return agrees;
}
public void setAgrees(Integer agrees) {
this.agrees = agrees;
}
public Integer getThanks() {
return thanks;
}
public void setThanks(Integer thanks) {
this.thanks = thanks;
}
public Integer getAsks() {
return asks;
}
public void setAsks(Integer asks) {
this.asks = asks;
}
public Integer getAnswers() {
return answers;
}
public void setAnswers(Integer answers) {
this.answers = answers;
}
public Integer getPosts() {
return posts;
}
public void setPosts(Integer posts) {
this.posts = posts;
}
public Integer getFollowees() {
return followees;
}
public void setFollowees(Integer followees) {
this.followees = followees;
}
public Integer getFollowers() {
return followers;
}
public void setFollowers(Integer followers) {
this.followers = followers;
}
public String getHashid() {
return hashid;
}
public void setHashid(String hashid) {
this.hashid = hashid == null ? null : hashid.trim();
}
}ScoreResult.java
package com.snailclimb.bean;
/**
* 圆饼图展示的时候需要使用到的对象
*
* @author Snailclimb
*
*/
public class ScoreResult {
public int value; // 点赞数
public String name;// 人名
public float getValue() {
return value;
}
public void setValue(int value) {
this.value = value;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public ScoreResult(int value, String name) {
super();
this.value = value;
this.name = name;
}
@Override
public String toString() {
return "ScoreResult [value=" + value + ", name=" + name + "]";
}
}
3.2 dao层
UserMapper.java
package com.snailclimb.dao;
import java.util.List;
import com.snailclimb.bean.User;
public interface UserMapper {
public List selecAgreesTop10();
}
UserMapper.xml
3.3 service层
UserService.java
package com.snailclimb.service;
import java.util.List;
import com.snailclimb.bean.User;
public interface UserService {
public List selecAgreesTop10();
}
3.4 service实现层
UserServiceImpl.java
package com.snailclimb.service.impl;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import com.snailclimb.bean.User;
import com.snailclimb.dao.UserMapper;
import com.snailclimb.service.UserService;
@Service
public class UserServiceImpl implements UserService {
@Autowired
private UserMapper userMapper;
@Override
public List selecAgreesTop10() {
return userMapper.selecAgreesTop10();
}
}
3.5 controller层
package com.snailclimb.controller;
import java.util.ArrayList;
import java.util.List;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import com.snailclimb.bean.ScoreResult;
import com.snailclimb.bean.User;
import com.snailclimb.service.UserService;
@Controller
@RequestMapping(value = "/echarts")
public class UserController {
@Autowired
private UserService userService;
/**
* 折线图和直方图
*
* @return
*/
@RequestMapping(value = "/agreeLineAndBar")
@ResponseBody
public List getAgreesTop10() {
return userService.selecAgreesTop10();
}
/**
* 圆饼图
*
* @return
*/
@RequestMapping(value = "/agreePie")
@ResponseBody
public List getAgreesTop10T() {
List list = userService.selecAgreesTop10();
List results = new ArrayList();
for (User user : list) {
ScoreResult scoreResult = new ScoreResult(user.getAgrees(), user.getUsername());
results.add(scoreResult);
}
return results;
}
// 跳转到圆饼图展示
@RequestMapping(value = "/intoagreePie")
public String intoagreePie() {
return "agreePie";
}
// 跳转到折线图和直方图
@RequestMapping(value = "/intoagreeLineAndBar")
public String Index() {
return "agreeLineAndBar";
}
} JSP页面
由于JSP页面代码过多，大家可以直接去我上传在Github的源码上拷贝。下面我只贴一下Ajax请求的代码。
下面以圆饼图为例，看看如何通过Ajax请求获取数据动态填充
对应的Controller层代码：
/**
* 圆饼图
*/
@RequestMapping(value = "/agreePie")
@ResponseBody
public List getAgreesTop10T() {
//查询数据
List list = userService.selecAgreesTop10();
List results = new ArrayList();
//将数据添加到results中
for (User user : list) {
ScoreResult scoreResult = new ScoreResult(user.getAgrees(), user.getUsername());
results.add(scoreResult);
}
//返回results
return results;
}
后台返回都是JSON格式的数据，如下图所示：
四 总结
这里只是以知乎赞同数TOP10为例子，带着大家学习了SSM环境的搭建以及代码的编写，代码中有很详细的注释。
通过本例子，大家完全可以自己做一个知乎粉丝数TOP、知乎感谢数TOP10等等例子出来。
另外本例子知识演示了圆饼图、折线图、柱状图的使用，大家可以自己去Echarts官网深入学习。
最后，本项目只是一个演示，还有很多需要优化的地方。比如可以使用redis来做缓存提高查询速度、可以创建索引提高查询速度或者直接将查询到的数据缓存下来等等方法来提高查询速度。
如果想要获取更多我的原创文章，欢迎关注我的微信公众号:" Java面试通关手册" 。无套路，希望能与您共同进步，互相学习。