列车信息获取(一)——数据总览
列车信息获取(一)——数据总览
第一次写博客，不知道怎么写。主要写的就是通过script的跨域访问去访问12306对外暴露的接口。然后取得车次信息并通过SSM框架保存在数据库中
开发环境:
JDK1.8+idea+SSM+httpclient4.5.3+Mysql5.7.16
获得的数据如下：
获得的数据大约有9000条。
1. 获得数据源的URL
这是获得12306上所有的车次信息。进入12306网站的车次查询页面.页面会自动加载信息,通过开发者模式获得URL.点开响应时可能会因为浏览器变慢.
数据连接
2. 搭建开发环境
数据获取后要保存到数据库中,所以采用idea搭建SSM框架,实际是初学只会SSM.具体搭建就不展示了,博客上很多搭建SSM环境的教程.附上现在需要的pom文件.
<properties>
<httpclient.version>4.5.3httpclient.version>
<druid.version>1.0.18druid.version>
<commons-lang3.version>3.4commons-lang3.version>
<commons-net.version>3.3commons-net.version>
<spring.version>4.3.4.RELEASEspring.version>
<mybatis.version>3.4.1mybatis.version>
<junit.version>4.11junit.version>
<slf4j.version>1.7.21slf4j.version>
<mysql.version>5.1.40mysql.version>
<jackson.version>2.7.3jackson.version>
<project.build.sourceEncoding>UTF-8project.build.sourceEncoding>
properties>
<dependencies>
<dependency>
<groupId>com.google.code.gsongroupId>
<artifactId>gsonartifactId>
<version>2.7version>
dependency>
<dependency>
<groupId>com.alibabagroupId>
<artifactId>druidartifactId>
<version>${druid.version}version>
dependency>
<dependency>
<groupId>mysqlgroupId>
<artifactId>mysql-connector-javaartifactId>
<version>${mysql.version}version>
dependency>
<dependency>
<groupId>commons-dbcpgroupId>
<artifactId>commons-dbcpartifactId>
<version>1.4version>
dependency>
<dependency>
<groupId>org.mybatisgroupId>
<artifactId>mybatisartifactId>
<version>${mybatis.version}version>
dependency>
<dependency>
<groupId>org.mybatisgroupId>
<artifactId>mybatis-springartifactId>
<version>1.3.0version>
dependency>
<dependency>
<groupId>org.mybatis.generatorgroupId>
<artifactId>mybatis-generator-coreartifactId>
<version>1.3.2version>
dependency>
<dependency>
<groupId>org.mybatis.generatorgroupId>
<artifactId>mybatis-generator-maven-pluginartifactId>
<version>1.3.5version>
dependency>
<dependency>
<groupId>org.apache.httpcomponentsgroupId>
<artifactId>httpclientartifactId>
<version>${httpclient.version}version>
dependency>
<dependency>
<groupId>org.apache.httpcomponentsgroupId>
<artifactId>httpcoreartifactId>
<version>4.4.4version>
dependency>
<dependency>
<groupId>com.github.pagehelpergroupId>
<artifactId>pagehelperartifactId>
<version>${pagehelper.version}version>
dependency>
<dependency>
<groupId>com.github.jsqlparsergroupId>
<artifactId>jsqlparserartifactId>
<version>0.9.5version>
dependency>
<dependency>
<groupId>junitgroupId>
<artifactId>junitartifactId>
<version>${junit.version}version>
<scope>testscope>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-aopartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-beansartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-coreartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-contextartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-expressionartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-aspectsartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-webartifactId>
<version>4.3.4.RELEASEversion>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-instrumentartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-instrument-tomcatartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-jdbcartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-jmsartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-ormartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-oxmartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-testartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-txartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-webmvcartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-webmvc-portletartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.springframeworkgroupId>
<artifactId>spring-websocketartifactId>
<version>${spring.version}version>
dependency>
<dependency>
<groupId>org.slf4jgroupId>
<artifactId>slf4j-apiartifactId>
<version>${slf4j.version}version>
dependency>
<dependency>
<groupId>org.slf4jgroupId>
<artifactId>slf4j-log4j12artifactId>
<version>1.7.21version>
dependency>
<dependency>
<groupId>log4jgroupId>
<artifactId>log4jartifactId>
<version>1.2.17version>
dependency>
<dependency>
<groupId>com.alibabagroupId>
<artifactId>fastjsonartifactId>
<version>1.2.7version>
dependency>
<dependency>
<groupId>com.fasterxml.jackson.coregroupId>
<artifactId>jackson-coreartifactId>
<version>2.7.3version>
dependency>
<dependency>
<groupId>com.fasterxml.jackson.coregroupId>
<artifactId>jackson-annotationsartifactId>
<version>2.7.3version>
dependency>
<dependency>
<groupId>com.fasterxml.jackson.coregroupId>
<artifactId>jackson-databindartifactId>
<version>2.7.3version>
dependency>
<dependency>
<groupId>org.apache.commonsgroupId>
<artifactId>commons-lang3artifactId>
<version>${commons-lang3.version}version>
dependency>
<dependency>
<groupId>commons-netgroupId>
<artifactId>commons-netartifactId>
<version>${commons-net.version}version>
dependency>
<dependency>
<groupId>commons-iogroupId>
<artifactId>commons-ioartifactId>
<version>2.4version>
dependency>
<dependency>
<groupId>commons-codecgroupId>
<artifactId>commons-codecartifactId>
<version>1.10version>
dependency>
dependencies>3. 获取数据
上面说到了从12306获得数据URL的格式为var train_list ={“2017-07-31”:{“D”:[{},{}]……,”G”:[{}{}]}},正因为如此我们可以使用script跨域访问.想了解的可以看: 通过script标签跨域、ajax跨域
<script src="https://kyfw.12306.cn/otn/resources/js/query/train_list.js?scriptVersion=1.0">script>然后我们就可以在代码中使用上面JS中的train_list.train_list的数据是json格式.可以直接用jquery遍历.
下面附上整个HTML的代码:
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
<meta http-equiv="Access-Control-Allow-Origin" content="*">
<title>车次获取title>
<link rel="stylesheet" href="/css/bootstrap.min.css"/>
<link rel="stylesheet" href="/bootstrap-table/bootstrap-table.css"/>
<script src="/js/jquery-2.2.3.min.js">script>
<script src="/js/bootstrap.min.js">script>
<script src="/js/vue.min.js">script>
<script src="https://kyfw.12306.cn/otn/resources/js/query/train_list.js?scriptVersion=1.0">script>
<script>
$(function () {
var trainlist = train_list;
var vm = new Vue({
el: "#app",
data: {
traval: '',
namelist: [],
},
watch: {},
methods: {
getalltrainline: function () {
$.each(trainlist, function (i, v) {
if (i == "2017-07-31") {
$.each(v, function (tou, che) {
//CDGKOTZ
if (tou == vm.traval) {
$.each(che, function (k, va) {
$.ajax({
url: "/train/addtrainline",
type: "POST",
data: {
trainno: va.train_no,
traintype: va.station_train_code.substring(0, 1),
traincode: va.station_train_code.substring(0, va.station_train_code.indexOf('(')),
trainstart: va.station_train_code.substring(va.station_train_code.indexOf('(') + 1, va.station_train_code.indexOf('-')),
trainend: va.station_train_code.substring(va.station_train_code.indexOf('-') + 1, va.station_train_code.indexOf(')'))
}
});
})
}
})
}
})
},
}
})
})
script>
head>
<body>
<div class="container" id="app">
<div class="row clearfix">
<div class="col-md-12 column">
<h4>这个页面都是跟火车有关系的东西呢h4>
div>
<div class="col-md-3 column">
<label>名称CDGKOTZlabel>
<input type="text" class="form-control" placeholder="请输入开头,如C,G,D" v-model="traval">
<button type="button" class="btn btn-primary" v-on:click="getalltrainline()">开始添加车次数据
button>
div>
div>
div>
div>
body>
html>Controller代码:
@Controller
@RequestMapping("/train")
public class TrainController {
@Autowired
private TrainServiceImlp trainService;
@RequestMapping("/addtrainline")
@ResponseBody
public int allUnit(HttpServletRequest request) {
Trainline trainline = new Trainline();
trainline.setTrainno(request.getParameter("trainno"));
trainline.setTraintype(request.getParameter("traintype"));
trainline.setTraincode(request.getParameter("traincode"));
trainline.setTrainstart(request.getParameter("trainstart"));
trainline.setTrainend(request.getParameter("trainend"));
int i = trainService.addTrainline(trainline);
return i;
}
}service代码:
public interface ITrainService {
int addTrainline(Trainline trainline);
List getlines();
int addlinestations(List list);
}
@Service
public class TrainServiceImlp implements ITrainService {
@Resource
private TrainlineMapper trainlineMapper;
@Resource
private LinestationMapper linestationMapper;
public int addTrainline(Trainline trainline) {
return trainlineMapper.insert(trainline);
}
} 下面的mapper,mapper.xml和实体类都是MyBatis Generator自动生成的,为了篇幅就简要复制..不会的可以百度也可以参考作者的博客:MyBatis Generator 详解
TrainlineMapper:
public interface TrainlineMapper {
int insert(Trainline record);
}TrainlineMapper.xml:
<mapper namespace="com.jeasen.hrms.dto.TrainlineMapper" >
<resultMap id="BaseResultMap" type="com.jeasen.hrms.pojo.Trainline" >
<id column="trainid" property="trainid" jdbcType="INTEGER" />
<result column="trainno" property="trainno" jdbcType="VARCHAR" />
<result column="traintype" property="traintype" jdbcType="VARCHAR" />
<result column="traincode" property="traincode" jdbcType="VARCHAR" />
<result column="trainstart" property="trainstart" jdbcType="VARCHAR" />
<result column="trainend" property="trainend" jdbcType="VARCHAR" />
resultMap>
<insert id="insert" parameterType="com.jeasen.hrms.pojo.Trainline" >
insert into trainline (trainid, trainno, traintype,
traincode, trainstart, trainend
)
values (#{trainid,jdbcType=INTEGER}, #{trainno,jdbcType=VARCHAR}, #{traintype,jdbcType=VARCHAR},
#{traincode,jdbcType=VARCHAR}, #{trainstart,jdbcType=VARCHAR}, #{trainend,jdbcType=VARCHAR}
)
insert>
mapper>Pojo实体类:
package com.jeasen.hrms.pojo;
public class Trainline {
private Integer trainid;
private String trainno;
private String traintype;
private String traincode;
private String trainstart;
private String trainend;
public Integer getTrainid() {
return trainid;
}
public void setTrainid(Integer trainid) {
this.trainid = trainid;
}
public String getTrainno() {
return trainno;
}
public void setTrainno(String trainno) {
this.trainno = trainno == null ? null : trainno.trim();
}
public String getTraintype() {
return traintype;
}
public void setTraintype(String traintype) {
this.traintype = traintype == null ? null : traintype.trim();
}
public String getTraincode() {
return traincode;
}
public void setTraincode(String traincode) {
this.traincode = traincode == null ? null : traincode.trim();
}
public String getTrainstart() {
return trainstart;
}
public void setTrainstart(String trainstart) {
this.trainstart = trainstart == null ? null : trainstart.trim();
}
public String getTrainend() {
return trainend;
}
public void setTrainend(String trainend) {
this.trainend = trainend == null ? null : trainend.trim();
}
}最后附上sql语句:
SET FOREIGN_KEY_CHECKS=0;
DROP TABLE IF EXISTS `trainline`;
CREATE TABLE `trainline` (
`trainid` int(11) NOT NULL AUTO_INCREMENT,
`trainno` varchar(66) DEFAULT NULL,
`traintype` varchar(22) DEFAULT NULL,
`traincode` varchar(22) DEFAULT NULL,
`trainstart` varchar(22) DEFAULT NULL,
`trainend` varchar(22) DEFAULT NULL,
PRIMARY KEY (`trainid`)
) ENGINE=InnoDB AUTO_INCREMENT=9058 DEFAULT CHARSET=utf8;
第一次写博客,水平很菜,还请各位多多包涵.