================================================================================
update嵌套查询修改不生效,是因为Navicat版本不支持
UPDATE `A` SET prop_count =15 WHERE id in
(select id from `A` where user_id in (select id from `B` where data_id="xx"
and status=2 and prop_expire_time="9999-12-31 23:59:59"
order by update_time DESC LIMIT 1));
背景:SQL监控接入dpc,日期为云自带的函数,但在本地odps调试时候不可以走云函数,需要自己获取当前时间-1,格式为yyyymmdd 如20191213
mysql中的DATE_FORMAT(NOW(),'%Y-%m-%d')函数
前一天日期
DATE_FORMAT(adddate(now(),-1),'%Y%m%d')
1.格式:
DATE_FORMAT(date,format) 函数用于显示日期或时间数据的不同样式。
1.1参数:date 合法的日期;
format 最终输出的日期/时间;
2.参考:
DATE_FORMAT(NOW(),’%Y-%m-%d’) 格式转换
SELECT DATE_FORMAT(NOW(),'%Y-%m-%d') AS '日期'
输出格式为2019-12-12
如果需要20191212格式的日期,则
mysql> select date_format(current_timestamp, '%Y%m%d')
-> ;
+------------------------------------------+
| date_format(current_timestamp, '%Y%m%d') |
+------------------------------------------+
| 20191224 |
+------------------------------------------+
该死的odps不支持mysql函数
select DATEADD(GETDATE(), -1, 'dd') from database.table_name limit 1;
select to_char(DATEADD(GETDATE(), -1, 'dd'),'yyyymmdd') from database.table_name limit 1;
但是第二个SQL不行,因为我最后要的是20191212这种的而第一个函数转出来是20191212 15:11:20这种的格式导致结果显示为yy1212
原因:''错误导致 应为“” 换成下面的就对了
select to_char(dateadd(GETDATE(),-1,'dd'),"yyyymmdd") from database.table_name limit 1;
第二种SQL为
select replace(split_part(dateadd(from_unixtime(unix_timestamp()),-1,'dd')," ",1),"-","") from database.table_name
思考:mysql一个函数搞定,odps需要多个函数,比较慢和烦人~~不能只用平台自带的配置调用配置ds='${bizdate}'
1技术会废掉 2出去面试找不到工作 3不要用傻瓜式配置,离开平台将没得学习和提高,还是要多思考。本地调试写死日期不明智,因为数据表会保留一周的数据,考虑到可用性和易维护性,还是写成函数吧,不能图省事,要看的长远~
===========================================================================
背景:
业务线埋点业务复杂,正则埋点130条+,基于UT平台经常断开链接以及每次回归成本较大,线上无监控,等着BI T+1发现问题的时候就晚了,对大盘数据造成影响。且QA最了解搜索埋点的不同场景的字段,基于odps做了埋点的监控S1测试方式:
1. 搜索埋点曝光case 21 条,点击case 64 条
其中,曝光case 1条为1-多个卡片,点击case1条为多个卡片(114个case)
// 统计mysql中一个字段的value中K出现的次数,如value值为 K3-22,K8-9,K10-2查看详情-二级页,查找K出现的次数,每个K是一个case
SELECT widget_name, sum(LENGTH(widget_name) - LENGTH( REPLACE(widget_name,'K','')))
from table_name where event_id=2101 and pd_emp_id in ("11","12","13","14","15" ,"16") ORDER BY gmt_modified DESC
1.1 曝光case校验字段:soku_test_ab、engine、item_log、aaid、k、track_info、source_from、search_from
具体规则如下,示例:
track_info 下的 soku_test_ab 正则规则 [a-z]{1}
track_info 下的 engine 正则规则 \S*
track_info 下的 item_log 正则规则 \S*
track_info 下的 k 正则规则 \S*
track_info 下的 aaid 正则规则 [0-9a-z]{32}
track_info 下的 source_from 正则规则 (home|discover|vip)
track_info 下的 search_from 正则规则 ^[1-9]\d?|1[01]\d|1[01]\d|^10trackinfo不为空spm正则规则a2h0c.8166622.PhoneXXPictureTab\d∗.channeltab\d∗(;|,)∗scm正则规则20140669.search.rgroupset.filter§∗1.2点击case校验字段:sokutestab、engine、itemlog、aaid、k、trackinfo、spm、scm结果页:trackinfo下的sokutestab正则规则[a−z]1trackinfo下的engine正则规则§∗trackinfo下的itemlog正则规则§∗trackinfo下的sourcefrom正则规则(home|discover|vip)trackinfo下的searchfrom正则规则[1−9]\d?trackinfo不为空spm正则规则a2h0c.8166622.PhoneXXPictureTab\d∗.channeltab\d∗(;|,)∗scm正则规则20140669.search.rgroupset.filter§∗1.2点击case校验字段:sokutestab、engine、itemlog、aaid、k、trackinfo、spm、scm结果页:trackinfo下的sokutestab正则规则[a−z]1trackinfo下的engine正则规则§∗trackinfo下的itemlog正则规则§∗trackinfo下的sourcefrom正则规则(home|discover|vip)trackinfo下的searchfrom正则规则[1−9]\d?|^1[01]\d|10|10
track_info 不为空
spm 正则规则 a2h0c.8166622.PhoneXXProgramSeries_\d*.poster_\d*
scm 正则规则 20140669.search.\S*.\S*
默认页:
track_info 下的 soku_test_ab 正则规则 [a-z]{1}
track_info 下的 aaid 正则规则 [0-9a-zA-Z]{32}
track_info 下的 k 正则规则 \S*
track_info 正则规则 \S*
spm 正则规则 a2h0c.8166619.PhoneXXOperate.clearbutton
scm 正则规则 20140669.search.searcharea.clearbutton
点击埋点新增soku_test_ab、engine、item_log字段,完成100%
共85条case,128个字段
分工:
曝光埋点
点击埋点
测试手段:
每个迭代版本一灰前覆盖测试所有点击 曝光埋点
通过正则平台筛选业务线,勾选case生成测试方案,手动运行,生成测试报告(成功、失败、未运行的case)
测试完毕QA发出测试报告
埋点涉及到的各种平台
埋点日志平台:抓取埋点实时日志
XX:埋点正则case地址
FBI监控平台:关注埋点监控日报
埋点数据监控平台
OneData报警平台:全部点击曝光case已接入,但暂不支持正则报警,待RD完善
埋点接入正则case
埋点监控以及报警
埋点相关数据表调研
select alldata.allcnt, faildata.failcnt, round(faildata.failcnt*100.0/alldata.allcnt,4) as failratio
from(
( select count(*) as failcnt,'trackInfo' as question
from (
SELECT a.* ,
get_json_object(a.track_info,'$.show_q') as show_q ,
get_json_object(a.track_info,'$.search_q') as search_q
from
(
select *
FROM xx.xx
WHERE ds=20191216 and site='xx' and (device='android' )
and (original_spm='a2h0c.8166622.home.default' or
original_spm='a2h0c.8166619.xx.default')
and app_version>'8.0.0'
)a
)b
WHERE
b.req_id is null or trim(b.req_id)=''
or b.show_q is null or trim(b.show_q)=''
or b.search_q is null or trim(b.search_q)=''
or b.recext is NULL or trim(b.recext)=''
or b.aaid is NULL or trim(b.aaid)=''
or b.alginfo is null or trim(b.alginfo)=''
)faildata
LEFT JOIN (
select count(*) as allcnt, 'trackInfo' as question
FROM xx.xx
WHERE ds=20191210 and site='xx' and (device='android' )
and (original_spm='a2h0c.8166622.home.default' or
original_spm='a2h0c.8166619.xx.default' )
)alldata
on alldata.question=faildata.question
);
ROUND() 函数
ROUND 函数用于把数值字段舍入为指定的小数位数。
SQL ROUND() 语法
SELECT ROUND(column_name,decimals) FROM table_name
参数 | 描述 |
---|---|
column_name | 必需。要舍入的字段。 |
decimals | 必需。规定要返回的小数位数。 |
计算一个带条件和不带条件的比例,改为用sum(case when 带条件 then 1 else 0),count(1) 然后再把两个字段作除法
就是
SELECT
SUM(
CASE parent_id
WHEN 0 THEN
'00'
WHEN 1 THEN
'11'
ELSE
'OTHERS'
END ) AS parent_id_new
2019-12-18日改版
select spm, count(distinct case when aaid is null then aaid else null end) over () , count(k) ,
group by spm
sum(expo) over (partition by a.dim) expo_all
或者
a b c d
select
*, a/all as rate
from (
select count(if(error<>'-1',1,null)) as a
, count(1) as all
, count(if(error='a',1,null)) as a1
-- select sum(if(error<>'-1',1,0)),sum(1),
from (
select *,
case when a=null then 'a'
when b=null then 'b'
else '-1'
end as error
) a
)
优化前的SQL:
设计上:
mysql也支持聚合的时候加一些条件,不过一般都是数据分析师才会这么搞,或者BI统计的时候用。日常这么写SQL,要被DBA干死的。因为这样很消耗Mysql的cpu,计算也一般都很慢,在线业务跑这种SQL,那接口几秒钟能返回也是够快了,随便几个并发起来了,库都要被拖挂了。
业务上:SQL查询的是A or B or C or D为空的总和计算/total_log,接入监控报警,一旦报警,无法准确定位是哪个字段出错了,可能是A可能是B可能是C,因为计算的是总和出错率。到时候还需要把SQL粘贴到odps,逐一修改判断哪个字段为空
优化后的SQL:
分别计算A为空B为空C为空D为空的出错率,因为最终的监控只能监控一个字段,所以需要sum(*)输出一个值接入监控系统
校验1个场景下的10个字段,统计,占比,如果哪个字段漏掉了,排查的时候,只需要把监控SQL粘贴到odps去掉sum(*),清晰地看到ABCD各自的失败率占比。哪个字段为空,准确定位
不需要分多个规则配置,不需要分端,减少冗余无效的复制粘贴以及一堆规则的填写,精简化,报错明显
--odps sql
--********************************************************************--
--author:老子
--create time:2019-12-18 19:29:09
--********************************************************************--
-- select COUNT(spm_nullrate) OVER (),COUNT(scm_nullrate) over() from ( --计算的值占总数的比
-- select sum( case when COALESCE(spm,'')<>'' then 1 else 0 end) as spm_null, sum(1) scm_total,
-- select device, sum( case when COALESCE(spm,'')<>'' then 1 else 0 end) as spm_null,sum(1),
-- sum( case when spm is null or trim(spm)='' then 1 else 0 end) /sum(1) as spm_rate,
-- as可以省略不写 如下
-- select
-- sum( case when spm is null or trim(spm)='' then 1 else 0 end) as spm_null, sum(1) log_total,
-- round(sum( case when spm is null or trim(spm)='' then 1 else 0 end) /sum(1) ,4) spm_rate,
--以下SQL的as也可以去掉
-- //通用需求柏拉图必须存在的key的校验
-- public static String[] mustExistKeyOuter = {"spm", "scm"};
-- public static String[] mustExistKeyInner = {"soku_test_ab", "engine", "item_log", "aaid", "k", "source_from", "search_from"};
select device,
sum( case when spm is null or trim(spm)='' then 1 else 0 end) as spm_null, sum(1) as log_total,
round(sum( case when spm is null or trim(spm)='' then 1 else 0 end) /sum(1) ,4) as spm_rate,
sum( case when scm is null or trim(scm)='' then 1 else 0 end) as scm_null,
round(sum( case when scm is null or trim(scm)='' then 1 else 0 end) /sum(1),4) as scm_rate,
sum( case when track_info is null or trim(track_info)='' then 1 else 0 end) as track_info_null,
round(sum( case when track_info is null or trim(track_info)='' then 1 else 0 end) /sum(1),4) as track_info_rate,
sum(case when get_json_object(track_info,'$.soku_test_ab') is null or trim(get_json_object(track_info,'$.soku_test_ab'))='' then 1 else 0 end) as soku_test_ab_null,
round(sum( case when get_json_object(track_info,'$.soku_test_ab') is null or trim(get_json_object(track_info,'$.soku_test_ab'))='' then 1 else 0 end) /sum(1),4) as soku_test_ab_rate,
sum( case when get_json_object(track_info,'$.engine') is null or trim(get_json_object(track_info,'$.engine'))='' then 1 else 0 end) as engine_null,
round(sum( case when get_json_object(track_info,'$.engine') is null or trim(get_json_object(track_info,'$.engine'))='' then 1 else 0 end) /sum(1),4) as engine_rate,
sum( case when get_json_object(track_info,'$.item_log') is null or trim(get_json_object(track_info,'$.item_log'))='' then 1 else 0 end) as item_log_null,
round(sum( case when get_json_object(track_info,'$.item_log') is null or trim(get_json_object(track_info,'$.item_log'))=''then 1 else 0 end) /sum(1),4) item_log_rate,
sum( case when get_json_object(track_info,'$.aaid') is null or trim(get_json_object(track_info,'$.aaid'))='' then 1 else 0 end) as aaid_null,
round(sum( case when get_json_object(track_info,'$.aaid') is null or trim(get_json_object(track_info,'$.aaid'))='' then 1 else 0 end) /sum(1),4) as aaid_rate,
sum( case when get_json_object(track_info,'$.k') is null or trim(get_json_object(track_info,'$.k'))='' then 1 else 0 end) as k_null,
round(sum( case when get_json_object(track_info,'$.k') is null or trim(get_json_object(track_info,'$.k'))='' then 1 else 0 end) /sum(1),4) as k_rate,
sum( case when get_json_object(track_info,'$.source_from') is null or trim(get_json_object(track_info,'$.source_from'))='' then 1 else 0 end) as source_from_null,
round(sum( case when get_json_object(track_info,'$.source_from') is null or trim(get_json_object(track_info,'$.source_from'))='' then 1 else 0 end) /sum(1),4) as source_from_rate,
sum( case when get_json_object(track_info,'$.search_from') is null or trim(get_json_object(track_info,'$.search_from'))='' then 1 else 0 end) as search_from_null,
round(sum( case when get_json_object(track_info,'$.search_from') is null or trim(get_json_object(track_info,'$.search_from'))='' then 1 else 0 end) /sum(1),4) as search_from_rate
from database.table_name(库名.表名)
WHERE ds=20191216 and hh=07 and site='xx' and (device='android' or device='iphone' )
and spm like '%a2h0c.8166619.hotkeyword%' and original_spm like '%a2h0c.8166619.xx%'
and original_spm!='a2h0c.8166619.xxhistory.open'
and original_spm!='a2h0c.8166619.xxhistory.close' GROUP BY device;
-- group by spm,scm,track_info,engine);
运行结果:
由于监控系统规则只能有一个字段报警,故此对所有字段的failrate求和,如需排查问题,则去掉
select device,round(spm_rate+scm_rate+track_info_rate+soku_test_ab_rate+engine_rate+item_log_rate+aaid_rate+k_rate+source_from_rate+search_from_rate ,4)
as total_failrate FROM (
最终接入监控的SQL如下
select device,round(spm_rate+scm_rate+track_info_rate+soku_test_ab_rate+engine_rate+item_log_rate+aaid_rate+k_rate+source_from_rate+search_from_rate ,4)
as total_failrate FROM (
select
sum( case when spm is null or trim(spm)='' then 1 else 0 end) as spm_null, sum(1) as log_total,
round(sum( case when spm is null or trim(spm)='' then 1 else 0 end) /sum(1) ,4) as spm_rate,
sum( case when scm is null or trim(scm)='' then 1 else 0 end) as scm_null,
round(sum( case when scm is null or trim(scm)='' then 1 else 0 end) /sum(1),4) as scm_rate,
sum( case when track_info is null or trim(track_info)='' then 1 else 0 end) as track_info_null,
round(sum( case when track_info is null or trim(track_info)='' then 1 else 0 end) /sum(1),4) as track_info_rate,
sum(case when get_json_object(track_info,'$.soku_test_ab') is null or trim(get_json_object(track_info,'$.soku_test_ab'))='' then 1 else 0 end) as soku_test_ab_null,
round(sum( case when get_json_object(track_info,'$.soku_test_ab') is null or trim(get_json_object(track_info,'$.soku_test_ab'))='' then 1 else 0 end) /sum(1),4) as soku_test_ab_rate,
sum( case when get_json_object(track_info,'$.engine') is null or trim(get_json_object(track_info,'$.engine'))='' then 1 else 0 end) as engine_null,
round(sum( case when get_json_object(track_info,'$.engine') is null or trim(get_json_object(track_info,'$.engine'))='' then 1 else 0 end) /sum(1),4) as engine_rate,
sum( case when get_json_object(track_info,'$.item_log') is null or trim(get_json_object(track_info,'$.item_log'))='' then 1 else 0 end) as item_log_null,
round(sum( case when get_json_object(track_info,'$.item_log') is null or trim(get_json_object(track_info,'$.item_log'))=''then 1 else 0 end) /sum(1),4) item_log_rate,
sum( case when get_json_object(track_info,'$.aaid') is null or trim(get_json_object(track_info,'$.aaid'))='' then 1 else 0 end) as aaid_null,
round(sum( case when get_json_object(track_info,'$.aaid') is null or trim(get_json_object(track_info,'$.aaid'))='' then 1 else 0 end) /sum(1),4) as aaid_rate,
sum( case when get_json_object(track_info,'$.k') is null or trim(get_json_object(track_info,'$.k'))='' then 1 else 0 end) as k_null,
round(sum( case when get_json_object(track_info,'$.k') is null or trim(get_json_object(track_info,'$.k'))='' then 1 else 0 end) /sum(1),4) as k_rate,
sum( case when get_json_object(track_info,'$.source_from') is null or trim(get_json_object(track_info,'$.source_from'))='' then 1 else 0 end) as source_from_null,
round(sum( case when get_json_object(track_info,'$.source_from') is null or trim(get_json_object(track_info,'$.source_from'))='' then 1 else 0 end) /sum(1),4) as source_from_rate,
sum( case when get_json_object(track_info,'$.search_from') is null or trim(get_json_object(track_info,'$.search_from'))='' then 1 else 0 end) as search_from_null,
round(sum( case when get_json_object(track_info,'$.search_from') is null or trim(get_json_object(track_info,'$.search_from'))='' then 1 else 0 end) /sum(1),4) as search_from_rate
,device
from database.table_name
WHERE ds=20191216 and hh=07 and site='xx' and (device='android' or device='iphone' )
and spm like '%a2h0c.8166619.hotkeyword%' and original_spm like '%a2h0c.8166619.xx%'
and original_spm!='a2h0c.8166619.xx.open'
and original_spm!='a2h0c.8166619.xx.close' GROUP BY device);
运行结果
==========================================================================================
捞取线上用户操作行为的日志,做监控用
String sql = "SELECT content FROM table where ds=\'" + currentDay + "\' and content like \"%mobile_multi%\" and (content like '%越狱兔%' or tolower(content) like '%liudehua%' or content like '%朱一龙%' or content like '%新白%' or content like '%角斗%' ) and content like \"%XXoNoi/EHHcDAOzWV3UxLzWW%\" and content like \"%8.2.7.2%\" ORDER BY rowkey DESC limit 100;";
SQL 字母大小写转换函数UPPER()、UCASE()、LOWER()和LCASE()
1、SQL UPPER()函数
SQL upper()函数字母大小写转换函数,将字母转成大写 - 返回字符串str,根据当前字符集映射的所有字符更改为大写。
SQL> SELECT UPPER('Allah-hus-w3cschool');
+---------------------------------------------------------+
| UPPER('Allah-hus-w3cschool') |
+---------------------------------------------------------+
| ALLAH-HUS-W3CSCHOOL |
+---------------------------------------------------------+
1 row in set (0.00 sec)
1
2
3
2. SQL LOWER()函数
SQL lower()字母大小写转换函数,将字母转成小写 - 返回根据当前字符集映射所有字符改变为小写,即返回小写的字符串。
SQL> SELECT LOWER('W3CSCHOOL');
+---------------------------------------------------------+
| LOWER('W3CSCHOOL') |
+---------------------------------------------------------+
| w3cschool |
+---------------------------------------------------------+
1 row in set (0.00 sec)
1
2
3
3. SQL UCASE() 函数
UCASE() 函数把字段的值转换为大写。
(1) SQL UCASE() 语法
SELECT UCASE(column_name) FROM table_name;
1
(2)用于 SQL Server 的语法
SELECT UPPER(column_name) FROM table_name;
1
4. SQL LCASE() 函数
- LCASE() 函数把字段的值转换为小写
(1)SQL LCASE() 语法
SELECT LCASE(column_name) FROM table_name;
1
(2)用于 SQL Server 的语法
SELECT LOWER(column_name) FROM table_name;
1 方式一
清空表数据并且将主键自增长从1开始(1.先清空表数据2.在把表的自增长设置为1)
DELETE FROM table_name;
ALTER TABLE table_name AUTO_INCREMENT=1;
方式2
使用truncate清空全部数据主键自增长是从1开始(效率更高)
truncate table table_name;
===============================================================================================
sql查询条件,DDL定义的字段为int类型,查询SQL的时候不用加 "" varchar类型要加。如果是int的类型加了引号效率会变低,如果条件中int类型全加了引号,查询时间上,不差多少,看不出来
原因:int类型加了"" ,查询的时候,不会走索引,虽然都存的数字,int 加了引号只是多了个类型转换,消耗转化性能,索引还是能用上,varchar 不加就是灾难。但是如果是in(1,2,"3") 一堆数字,其中有的有引号,有的没有引号,会有影响。int不要加"" ,不管出现在什么查询中。主要是机器要识别,int加上引号、varchar类型不加引号,机器想识别,就要隐式转换。转换就会带来性能问题。原理:Mysql会将传入的string (要赋值给int字段的) 从左到右的第一个非数值开始, 将后面的字符串转成0,在和数值类型相加
58到家MySQL军规升级版 https://mp.weixin.qq.com/s/YfCORbcCX1hymXBCrZbAZg
=============================================================================
mysql查询sql 查询结果中,控件名称不包含某个字符K
// 查询正则case平台,控件名称不包含字符K的搜索数据,唯一ID spmCD位去重,删除多余的正则。天多的case,打K标签,里面原有case有soku,小写的k
SELECT id,widget_name,gmt_modified,spma,spmb,page_name FROM `XX` WHERE widget_name not in (select widget_name from `XX` where widget_name LIKE "%K%") and event_id= 2101 and pd_emp_id in ("11","12","13","14")
// MySQL的like查询是不区分大小写的。解决办法方法一(查询时,指定区分大小写),在like的后面加个binary,适用于表的结构不易改变的情况下
SELECT id,widget_name, gmt_modified,spma,spmb,unique_value,page_name
FROM `XX` WHERE widget_name not in (
select widget_name from `XX` where widget_name LIKE BINARY "%K%" and pd_emp_id in
("11","12","13","14")) and spma="a2h0c" and event_id ="2101";
上面使用_bin来区分大小写的,还可以使用
create table table_name (word VARCHAR(10)) CHARACTER SET latin1 COLLATE latin1_general_cs;
或者,在查询时指定collation
建表时:mysql> create table table_name (word VARCHAR(10)) CHARACTER SET latin1;
查询时:mysql> SELECT * FROM table_name WHERE word COLLATE latin1_bin LIKE 'F%'; 放在like前面
或:mysql> SELECT * FROM table_name WHERE word LIKE 'F%' COLLATE latin1_bin; 放在最后
或:mysql> SELECT * FROM case_test WHERE word LIKE 'F%' COLLATE latin1_general_cs;
参考文献:http://www.cnblogs.com/pinnasky/archive/2012/09/11/2680264.html
参考:
http://www.cnblogs.com/zi-xing/p/4298697.html
==============================================================================================
mysql 去除字段value值前后的空格:前空格ltrim 后空格rtrim 前后空格trim
UPDATE XX set page_name = trim(page_name) WHERE spma = 'a2h0c'
mysql> SELECT TRIM(' bar '); -- 默认删除前后空格
-> 'bar'
mysql> SELECT TRIM(LEADING ',' FROM ',,barxxx'); -- 删除指定首字符 如','
-> 'barxxx'
mysql> SELECT TRIM(BOTH ',' FROM ',,bar,,,'); -- 删除指定首尾字符
-> 'bar'
mysql> SELECT TRIM(TRAILING ',' FROM 'barxxyz,,');
-> 'barxxyz'
神奇了,去空格后还是这样,查了半天,最后感觉这不是空格,可能是tab键等,最后走update更新
// 查询page_searchhome没有空格的数据,查到的数量为8
SELECT id,widget_name,page_name,spma FROM `XX` WHERE widget_name LIKE "%K0%" and event_id= 2101 and page_name= "page_searchhome" ORDER BY id DESC;
// 查询page_searchhome和可能带其他非空格的字符的page_searchhome数据,查到的数量为9
SELECT id,widget_name,page_name,spma FROM `XX` WHERE widget_name LIKE "%K0%" and event_id= 2101 and spma = "a2h0c" and page_name not in ("page_searchresults") ORDER BY id DESC;
相差的数据是ID=1797
或1条SQL
SELECT id,widget_name,page_name,spma FROM `XX` WHERE widget_name LIKE "%K0%" and event_id= 2101
and spma = "a2h0c" and page_name not in ("page_searchresults", "page_searchhome")
排查是否为非空格
package com.alibaba.yksearch.util;
import java.nio.charset.Charset;
public class test {
public static void main(String[] args){
byte[] a=" ".getBytes(Charset.defaultCharset());
byte[] b="\t".getBytes(Charset.defaultCharset());
System.out.println(a);
System.out.println(b);
}
}
=======================================================================================
SELECT id,widget_name,page_name,spma FROM `XX` WHERE widget_name LIKE "%K0%"
and event_id=2101 and page_name not in ("page_searchhome") ORDER BY gmt_modified DESC;
SELECT id,widget_name,page_name,spma FROM `XX` WHERE widget_name LIKE "%K0%"
and event_id=2101 and page_name !="page_searchhome" ORDER BY gmt_modified DESC;
当性能问题要看具体语句,建议用not in,优化器对!=支持不太好。当一条数据的时候,!=不走索引。建议用not in ,尽量按照规范写sql
explain模拟优化器执行SQL语句,在5.6以及以后的版本中,除过select,其他比如insert,update和delete均可以使用explain查看执行计划,从而知道mysql是如何处理sql语句,分析查询语句或者表结构的性能瓶颈。
作用
1、表的读取顺序
2、数据读取操作的操作类型
3、哪些索引可以使用
4、哪些索引被实际使用
5、表之间的引用
6、每张表有多少行被优化器查询
UPDATE XX set page_name = trim(page_name) WHERE spma = 'a2h0c'
select page_name,spmb FROM XX WHERE spma = "a2h0c"
单引号和双引号的区别:单引号是SQL官方的用法,双引号是后来引入的支持。数据库都认为是字符,没有太大的区别。
// 查询搜索业务点击埋点的全部正则case,带K标签
SELECT id,widget_name, page_name,priority,spma,spmb,unique_value,page_name,gmt_modified FROM `XX` WHERE widget_name LIKE BINARY "%K%" and pd_emp_id in ("11", "12", "13", "14", "15", "16") and spma="a2h0c" and event_id ="2101";
// 查询搜索业务点击埋点的全部正则case,前10个带K标签大流量搜索类型数据
SELECT id,widget_name, page_name,priority,spma,spmb,unique_value,page_name,gmt_modified
FROM `XX` WHERE widget_name regexp 'K0-|K-|K1-|K2-|K3-|K4-|K5-|K6-|K7-|K8-|K9-|K10-' and pd_emp_id in
("11", "12", "13", "14", "15", "16") and spma="a2h0c" and event_id ="2101";
// 将10个卡片设置为高优先级,1 - 低 2 - 中 3 -高
UPDATE `XX` SET priority=3 WHERE widget_name regexp 'K0-|K-|K1-|K2-|K3-|K4-|K5-|K6-|K7-|K8-|K9-|K10-' and pd_emp_id in
("11", "12", "13", "14", "15", "16") and spma="a2h0c" and event_id ="2101";
// 查询库.表中ddl的列名和注释
select column_name, column_comment from information_schema.columns where table_schema ='dbname' and
table_name = 'XX' ;
// 查看库.表的注释
select table_name,table_comment from information_schema.tables where table_schema = 'dbname' and table_name ='XX';
// 报警系统onedata接入柏拉图所需字段,搜索点击事件2101
SELECT event_id, spma, page_name, page_name_cn, id, widget_name, unique_value, priority, pd_emp_id, pd_emp_id
FROM `XX` WHERE pd_emp_id in ("11", "12", "13", "14", "15", "16") and event_id ="2101";
// 报警系统onedata接入柏拉图所需字段,搜索曝光事件2201
SELECT event_id, spma, page_name, page_name_cn, id, page_name_cn, unique_value, widget_name, pd_emp_id, pd_emp_id
FROM `XX` WHERE pd_emp_id in ("11", "12", "13", "14", "15", "16") and event_id ="2201";
// 统计mysql中一个字段的value中K出现的次数,如value值为 K3-22,K8-9,K10-2查看详情-二级页,查找K出现的次数,每个K是一个case
SELECT widget_name, sum(LENGTH(widget_name) - LENGTH( REPLACE(widget_name,'K','')))
from `plato_track_demand` where event_id=2101 and pd_emp_id in ("11", "12", "13", "14", "15", "16") ORDER BY gmt_modified DESC
SQL中并没有提供计算字符在字符串中出现的次数的函数,不过可以通过下面的思路来等价实现:首先计算字符串的长度,然后将字符串的所有“r”删掉,计算删掉“r”以后的字符串的长度,这两个长度之差就是字符“r”在字符串中出现的次数
参考博客 https://my.oschina.net/u/1866821/blog/672752
================================================================================================================================================================================================
监控系统问题排查-踩坑系列
1. 埋点监控概览
日志总行数 曝光抽样10% 命中率=命中样本数/日志总行数
其他的埋点都是抽样100%
灰度的所有埋点
2. 埋点监控日报:离线数据,只能看T-1的数据,即当天的数据看不了 。其他tab都是实时的
需求0为正则平台规则,其他[2][3]为其他平台的规则
一小时日志几万行
报表主要是BI要看的数据 QA主要关注日志
1.柏拉图正则如,PhoneSokuCast_\d*.portrait_\d* 在录入报警系统时需要转义为PhoneSokuCast_\\d*.portrait_\\d*
数据库会有特殊处理,字符串碰到 \ 会特殊处理
2.参考v6版本的模板,把分母的地方都加一个大于0判断,否则有没有监控到的埋点会报错
SELECT WM_CONCAT(',',point_id)
FROM (
SELECT event_id AS event_id
,page_name AS page_name
,point_id AS point_id
,unique_key AS unique_key
,SUM(passed_all_uv_603_520) AS passed_all_uv_603_520
,SUM(all_cnt_uv_603_520) AS all_cnt_uv_603_520
FROM (
SELECT t2.event_id
,t2.page_name
,t2.point_id
,t2.unique_key
,CASE WHEN SUM(CASE
WHEN t1.version='8.1.0' THEN t1.passed_all_uv ELSE 0 END )>0 THEN SUM(CASE
WHEN t1.version='8.1.0.1' THEN t1.passed_all_uv ELSE 0 END )/SUM(CASE
WHEN t1.version='8.1.0' THEN t1.passed_all_uv
ELSE 0
END ) ELSE 0 END AS passed_all_uv_603_520
,CASE WHEN SUM(CASE
WHEN t1.version='8.1.0' THEN t1.all_cnt_uv ELSE 0 END )>0 THEN SUM(CASE
WHEN t1.version='8.1.0.1' THEN t1.all_cnt_uv ELSE 0 END )/SUM(CASE
WHEN t1.version='8.1.0' THEN t1.all_cnt_uv
ELSE 0
END ) ELSE 0 END AS all_cnt_uv_603_520
FROM (
SELECT ds
,SPLIT_PART(rowkey,'\004',2) AS point_id
,SPLIT_PART(rowkey,'\004',8) AS version
,CASE WHEN all_uv>0 THEN passed_uv/all_uv
ELSE 0
END AS passed_all_uv
,CASE WHEN all_uv>0 THEN all_cnt/all_uv
ELSE 0
END AS all_cnt_uv
FROM db.table2
WHERE ds = '20190815'
AND SPLIT_PART(rowkey,'\004',3) = '0'
AND SPLIT_PART(rowkey,'\004',4) = '0'
AND SPLIT_PART(rowkey,'\004',5) = '23569910'
AND SPLIT_PART(rowkey,'\004',8) IN ('8.1.0','8.1.0.1')
AND SPLIT_PART(rowkey,'\004',9) = '20190815'
AND SPLIT_PART(rowkey,'\004',10) = '0'
) t1
JOIN (
SELECT point_id
,event_id
,page_name
,unique_key
FROM db.table1
WHERE ds = '20190815'
AND demand_status = '1'
AND (demand_id IS NULL OR demand_id = '0')
) t2
ON t1.point_id = t2.point_id
GROUP BY t2.event_id
,t2.page_name
,t2.point_id
,t2.unique_key
) a
GROUP BY event_id
,page_name
,point_id
,unique_key
) a
WHERE CONCAT('#-_-!#',COALESCE(event_id, ''),'#-_-!#',COALESCE(page_name, '')
统计报警查询的case数目即point_id,oneData里的实例管理-查看日志
用odps的函数,多行id以一行展示,逗号拆分
WM_CONCAT(',',point_id)
4. idb中正则平台caseID统计
SELECT group_concat(id) FROM `XX` WHERE event_id in(2101,2201) and pd_emp_id in
("11", "12", "13", "14", "15", "16") ORDER BY gmt_modified DESC;
5. odps和idb的数据对比,定位为什么odps日志没用监控到正则平台case
package com.alibaba.yksearch.util;
import java.util.*;
public class idDiff {
public static void main(String[] args) {
String odps = "3513,2010,3305,3552,3495,1132,1602,1615,1797,1799,1801,1802,2283,1850,3548,3549,3550,1846,1881,1882,1901,1903,1927,1938,1962,1968,2021,2309,2557,3283,3286,3289,3292,3293,3298,3299,3302,3306,3307,3324,3326,3327,3328,3329,3353,3355,3357,3514,3530,3535,3537,3541,3542,3543,3553,3555,3532,3533,3475,3476,3477,3479,3483,3484,3489,3496,3498,3501,3502,3504,3511,3515,3518,3575";
String idb = "1132,1602,1615,1797,1799,1800,1801,1802,1846,1850,1881,1882,1888,1901,1903,1927,1938,1962,1963,1965,1968,2010,2015,2021,2283,2309,2557,3283,3286,3289,3292,3293,3298,3299,3302,3305,3306,3307,3311,3324,3326,3327,3328,3329,3353,3355,3357,3475,3476,3477,3479,3483,3484,3489,3490,3495,3496,3498,3501,3502,3504,3511,3512,3513,3514,3515,3518,3522,3529,3530,3532,3533,3535,3537,3538,3539,3541,3542,3543,3544,3548,3549,3550,3552,3553,3555,3573,3575";
String[] arrayopds = odps.split(",");
String[] arrayidb = idb.split(",");
List listodps = new ArrayList<>(Arrays.asList(arrayopds));
List listidb = new ArrayList<>(Arrays.asList(arrayidb));
listidb.removeAll(listodps);
System.out.print(listidb);
}
}
运行结果,[1800, 1888, 1963, 1965, 2015, 3311, 3490, 3512, 3522, 3529, 3538, 3539, 3544, 3573]
去正则平台一条条排查这些caseID
================================================================================================================================================================================================
更新107*3条case,当然技术方法解决。最骚的是没有接口文档,只能通过SQL搞定啦~~~
// 1先查询plato_track_demand的caseID(需求ID),找到对应的rule_id;2再从plato_track_rule找对应的id;3再从plato_track_expression找对应的正则表达式
// case - 一级正则- 二级正则,3张表对应关系如下
SELECT * FROM `plato_track_expression` WHERE rule_id in
(SELECT rule_id FROM `plato_track_demand` WHERE pd_emp_id in ("1","2","3","4") and common_rule_id in
(SELECT id from `plato_track_rule`)) ORDER BY rule_id DESC
// 拆分123每步查询的SQL如下
SELECT * FROM `plato_track_demand` WHERE pd_emp_id in ("1","2","3","4") and rule_id in (SELECT id from `plato_track_rule`)
SELECT * from `plato_track_rule`
SELECT * FROM `plato_track_expression` WHERE rule_id in (SELECT id from `plato_track_rule`)
SELECT * FROM `plato_track_expression` WHERE rule_id in
(SELECT rule_id FROM `plato_track_demand` WHERE pd_emp_id in ("1","2","3","4")) ORDER BY rule_id DESC
SELECT * FROM `plato_track_expression` as a inner join `plato_track_rule` as b
on a.rule_id=b.id ORDER BY a.id desc
SELECT * FROM `plato_track_expression` WHERE rule_id in (SELECT id from `plato_track_rule`)
// 搜索业务点击曝光埋点每条case的track_info下插入item_log算法信息字段
INSERT INTO `plato_track_expression` (
expression_key,
expression_mark,
expression_val,
expression_desc,
rule_id,
expression_child_key)
( SELECT "track_info", "4", "\\S*", "算法信息item_log", id, "item_log" FROM `plato_track_rule` WHERE creator_id IN ("1","2","3","4"))
// 搜索业务点击曝光埋点每条case的track_info下插入engine算法信息字段
INSERT INTO `plato_track_expression` (
expression_key,
expression_mark,
expression_val,
expression_desc,
rule_id,
expression_child_key)
( SELECT "track_info", "4", "\\S*", "算法信息engine", id, "engine" FROM `plato_track_rule` WHERE creator_id IN ("1","2","3","4"))
// 搜索业务点击曝光埋点每条case的track_info下插入soku_test_ab算法信息字段
INSERT INTO `plato_track_expression` (
expression_key,
expression_mark,
expression_val,
expression_desc,
rule_id,
expression_child_key)
( SELECT "track_info", "4", "[a-z]{1}", "算法信息soku_test_ab", id, "soku_test_ab" FROM `plato_track_rule` WHERE creator_id IN ("1","2","3","4"))
// 修改107*3 条case之前,先修改【搜索业务-查询埋点管理-规则】通用正则数目,19条
SELECT * FROM `plato_track_rule` WHERE creator_id in ("1","2","3","4") and rule_type=1
ORDER BY id DESC LIMIT 100
// 查询【埋点管理-规则-验证规则明细】总的正则数目,182条,每条8-10条验证规则明细
SELECT * FROM `plato_track_expression` WHERE rule_id in
(SELECT id FROM `plato_track_rule` WHERE creator_id in ("1","2","3","4") and name IS NOT NULL and rule_type=1
ORDER BY id DESC ) ORDER BY id DESC
// 搜索业务点击曝光埋点【埋点管理-规则-验证规则明细】每个通用规则下插入3个算法字段,算法信息soku_test_ab、算法信息engine、算法信息item_log,但此SQL有问题。。。找不到原因
INSERT INTO `plato_track_expression` (
expression_key,
expression_mark,
expression_val,
expression_desc,
rule_id,
expression_child_key)
( SELECT "track_info", "4", "\\S*", "算法信息item_log", DISTINCT(rule_id), "item_log" FROM `plato_track_expression` WHERE rule_id in
(SELECT id FROM `plato_track_rule` WHERE creator_id in ("1","2","3","4") and name IS NOT NULL
ORDER BY id DESC ) ORDER BY id DESC )
// 查询搜索业务线通用规则正则表达式,所有的通用正则、case里的【验证规则明细】均在plato_track_expression,用rule_type区分,规则的种类,1公共 2普通,tinyint类型
SELECT DISTINCT(rule_id) FROM `plato_track_expression` WHERE rule_id in
(SELECT id FROM `plato_track_rule` WHERE creator_id in ("1","2","3","4") and rule_type=1
ORDER BY id DESC ) ORDER BY id DESC
SELECT * FROM `plato_track_expression` LIMIT 20
SELECT * FROM `plato_track_rule` WHERE creator_id in ("1","2","3","4") and name IS NOT NULL
0909新增SQL设计,统计埋点
SELECT UNIX_TIMESTAMP('2019-09-03')
SELECT * FROM `plato_track_check_report` WHERE work_id in("1","2","3","4") AND
UNIX_TIMESTAMP(gmt_create) > 1567440000 ORDER BY gmt_create DESC
================================================================================================================================================================================================
速算家长端
//type 类型 1 问答 2 文章
//status 状态 1 未审核 2 审核通过 3 审核不通过
SELECT * FROM edu_parent_circle_article WHERE type=1 AND parentId in (SELECT parent_id FROM susuan.edu_parent WHERE mobile=18711111222)
SELECT * FROM edu_parent_circle_article WHERE type=1 AND parentId in (SELECT parent_id FROM susuan.edu_parent WHERE mobile=18511871111)
//我的页面,我的问答
SELECT * FROM edu_parent_circle_article WHERE type=1 AND parentId=1147388
SELECT COUNT(*) FROM edu_parent_circle_article WHERE type=1 AND parentId=1147388
SELECT COUNT(*) FROM edu_parent_circle_reply WHERE parentId=1147388
SELECT * FROM edu_parent_circle_article WHERE (1) AND (status=2) ORDER BY articleId DESC LIMIT 20
SELECT * FROM edu_parent_circle_article WHERE circleId=1 AND type=1
SELECT * FROM edu_parent_circle_article WHERE circleId=2 AND type=1
SELECT * FROM edu_parent_circle_article WHERE circleId=2 AND type=2
//接口数据校验
//首页文章&问答列表接口数据
SELECT * FROM edu_parent_circle WHERE circleId=1
SELECT * FROM edu_parent_circle WHERE circleId=3
SELECT count(DISTINCT(parentId)),circleId FROM edu_parent_circle_article GROUP BY circleId
SELECT DISTINCT(parentId) FROM edu_parent_circle_article WHERE circleId=3
SELECT * FROM edu_parent_circle_article WHERE circleId=104 AND type=2
//答复列表
SELECT * FROM edu_parent_circle_reply WHERE parentId=1147388
SELECT * FROM edu_parent_circle_reply WHERE parentId=1147388
//家长提问和回答
SELECT * FROM edu_parent_circle_article WHERE parentId=1147388
SELECT * FROM edu_parent_circle_reply WHERE articleId=83
//浏览数,PV
UPDATE edu_parent_circle_article SET readNum=readNum+1 WHERE articleId=83
//一条SQL根据家长手机号查询学生手机号,3表查询,学生端,家长端,学生家长关联关系表
select a.* from wx_edu_student as a join edu_relate_parent_student as b on a.StudentID = b.student_id join edu_parent as c on b.parent_id = c.parent_id WHERE c.mobile = 18511876666
//一条SQL根据学生手机号查询家长手机号,3表查询,学生端,家长端,学生家长关联关系表
select a.* from edu_parent as a join edu_relate_parent_student as b on a.parent_id = b.parent_id join wx_edu_student as c on b.student_id = c.StudentID WHERE c.mobile = 18511871111
//多条SQL查询如下
SELECT * FROM wx_edu_student WHERE mobile=13701240090
SELECT * FROM wx_edu_student WHERE mobile=18953427368
SELECT * FROM edu_relate_parent_student WHERE student_id=16009040
SELECT * FROM edu_parent WHERE parent_id=1791379
SELECT * FROM susuan.edu_parent WHERE mobile=18511871111
SELECT * FROM edu_parent WHERE parent_id=1691754
SELECT * FROM edu_relate_parent_student WHERE parent_id=1691754
SELECT * FROM edu_parent WHERE parent_id=1726518
SELECT * FROM edu_game_product WHERE product_id=149
SELECT * FROM edu_coupon_product WHERE user_id=79841 AND product_id=149
SELECT * FROM edu_coupon_product WHERE product_id=149
//1后台附有答案审核通过用户发起的问答,无人回答,出现在已解答tab,测试小红点接口以及99+数据的测试
drop procedure if exists edu_parent_circle_article;
delimiter //
create procedure edu_parent_circle_article()
begin
declare num int;
SET num=1;
while num < 3 do
INSERT INTO edu_parent_circle_article (parentId, circleId, type, content,status,replyNum) VALUES (1147403, 1, 1, '0114QA发起问答',2,1);
SET num=num+1;
end while;
end
//
call edu_parent_circle_article();
//SQL存储过程--两张表写在一起,内循环。LAST_INSERT_ID取最后面的id,评论回复1对1
drop procedure if exists onetoone;
delimiter //
create procedure onetoone()
begin
declare a int;
declare num int;
declare num1 int;
SET num1=1;
while num1<=99 do
INSERT INTO edu_parent_circle_article (parentId, circleId, type, content,status,replyNum) VALUES (1147452, 1, 1, 'QAls测试循环1',2,1);
SET num1=num1+1;
set a = LAST_INSERT_ID();
SET num=1;
while num <= 1 do
INSERT INTO edu_parent_circle_reply (parentId, content,articleId) VALUES (1,'QALS循环的答复测试一对一评论',a);
SET num=num+1;
end while;
end while;
end
//
call onetoone();
//SQL存储过程--两张表写在一起,内循环。LAST_INSERT_ID取最后面的id,待解答
drop procedure if exists onetoone;
delimiter //
create procedure onetoone()
begin
declare a int;
declare num int;
declare num1 int;
SET num1=1;
while num1<=98 do
INSERT INTO edu_parent_circle_article (parentId, circleId, type, content,status,replyNum) VALUES (1147452, 1, 1, '待解答to QA春娜测试提问',1,1);
SET num1=num1+1;
set a = LAST_INSERT_ID();
SET num=1;
while num <= 1 do
INSERT INTO edu_parent_circle_reply (parentId, content,articleId) VALUES (1,'答复测试-带解答。。。。。',a);
SET num=num+1;
end while;
end while;
end
//
call onetoone();
//SQL存储过程--两张表写在一起,内循环。LAST_INSERT_ID取最后面的id,评论回复1对多.replyNum与内层循环次数保持一致。构造我的答复99条数据
drop procedure if exists onetomore;
delimiter //
create procedure onetomore()
begin
declare a int;
declare num int;
declare num1 int;
SET num1=1;
while num1<=1 do
INSERT INTO edu_parent_circle_article (parentId, circleId, type, content,status,replyNum) VALUES (1147452, 1, 1, 'QA to 春娜提问',2,99);
SET num1=num1+1;
set a = LAST_INSERT_ID();
SET num=1;
while num <= 99 do
INSERT INTO edu_parent_circle_reply (parentId, content,articleId) VALUES (1147388,'99条回复测试数量',a);
SET num=num+1;
end while;
end while;
end
//
call onetomore();
//查看该家长已解答的问答数目
SELECT * FROM edu_parent_circle_article WHERE parentId=1147388 AND STATUS=2
//插入99条问答数据,待解答数据测试--后台未审核。status 状态 1 未审核 2 审核通过 3 审核不通过
drop procedure if exists circle_article;
delimiter //
create procedure circle_article()
begin
declare num int;
SET num=1;
while num < 3 do
INSERT INTO edu_parent_circle_article (parentId,circleId,type,content,status,replyNum,readNum,isDelete,createTime,updateTime)
VALUES (1147388,1,1,'0112待解答Status为1 QA李杉-----测试,数据',1,0,0,0,'2018-01-12 15:47:14','2018-01-12 15:47:14');
SET num=num+1;
end while;
end
//
call circle_article();
//我的回答
drop procedure if exists my_reply;
delimiter //
create procedure my_reply()
begin
declare num int;
declare a int;
SET num=1;
set a = LAST_INSERT_ID();
while num < 3 do
INSERT INTO edu_parent_circle_reply (parentId, content,articleId) VALUES (1147388,'我的99条回复测试数量',a);
SET num=num+1;
end while;
end
//
call my_reply();
//查看该家长待解答的问答数目
SELECT * FROM edu_parent_circle_article WHERE parentId=1147388 AND STATUS=1
//DELETE FROM edu_parent_circle_article
//DELETE FROM edu_parent_circle_rel
//DELETE FROM edu_parent_circle_reply
//DELETE FROM edu_parent_circle_report
UPDATE edu_parent_circle_article SET readNum=readNum+1 WHERE articleId=455
DELETE FROM edu_parent_circle_article WHERE type=1
DELETE FROM edu_parent_circle_reply
//首页文章&问答列表。未加入任何圈子,从3个圈子筛选出5条,按发布时间倒叙。家长圈123
SELECT * FROM edu_parent_circle_article WHERE circleId=1 AND status=2 ORDER BY articleId DESC LIMIT 5
SELECT * FROM edu_parent_circle_article WHERE circleId=2 AND status=2 ORDER BY articleId DESC LIMIT 5
SELECT * FROM edu_parent_circle_article WHERE circleId=3 AND status=2 ORDER BY articleId DESC LIMIT 5
//加入家长圈
SELECT * FROM edu_parent_circle_rel WHERE parentId=1147474 ORDER BY id
SELECT * FROM edu_parent_circle_rel WHERE parentId=1147474 ORDER BY id
SELECT COUNT(*) FROM edu_parent_circle_article WHERE parentId=1147474 AND status=1 AND isDelete=0
SELECT COUNT(*) FROM edu_parent_circle_reply WHERE parentId=1147474 AND isDelete=0
SELECT COUNT(*) FROM edu_parent_circle_article WHERE type=2 AND circleId=1
SELECT COUNT(*) FROM edu_parent_circle_article WHERE type=2 AND circleId=2
SELECT COUNT(*) FROM edu_parent_circle_article WHERE type=2 AND circleId=3
//单个家长圈,文章列表 类型 (必填) // 1 问答 2 精读。返回20条数据
SELECT * FROM edu_parent_circle_article WHERE circleId=1 AND type=1 AND status=2 ORDER BY articleId DESC LIMIT 20
SELECT * FROM edu_parent_circle_article WHERE circleId=1 AND type=2 AND status=2 ORDER BY articleId DESC LIMIT 20
//提问回答,答复数校验
INSERT INTO edu_parent_circle_reply (parentId, articleId, content, toParentId, toReplyId) VALUES (1147474, 470, '给黑胡椒即可', 0, 0)
UPDATE edu_parent_circle_article SET replyNum=replyNum+1 WHERE articleId=470
//家长问答,每次返回20条数据测试
SELECT * FROM edu_parent_circle_article WHERE circleId=1 AND status=2 ORDER BY articleId DESC LIMIT 20
//我的回答,分页数据测试,nextFromId每20个replyId为1个返回,倒叙返回20条数据
SELECT * FROM edu_parent_circle_reply WHERE parentId=1147388 AND isDelete=0 AND replyId < 730 ORDER BY replyId DESC LIMIT 20
SELECT * FROM edu_parent_circle_reply WHERE parentId=1147388 AND isDelete=0 AND replyId < 710 ORDER BY replyId DESC LIMIT 20
SELECT * FROM edu_parent_circle_reply WHERE parentId=1147388 AND isDelete=0 AND replyId < 690 ORDER BY replyId DESC LIMIT 20
//文章阅读计数-articleId=692为问答,type 类型 1 问答 2 文章
UPDATE edu_parent_circle_article SET readNum=readNum+1 WHERE articleId=692
SELECT readNum FROM edu_parent_circle_article WHERE articleId=692
//文章阅读计数-articleId=443为文章,type 类型 1 问答 2 文章
UPDATE edu_parent_circle_article SET readNum=readNum+1 WHERE articleId=443
SELECT readNum FROM edu_parent_circle_article WHERE articleId=443
SELECT parent_id FROM susuan.edu_parent WHERE mobile=18511871111
//status 改为0 lastTime 改为30天之前的时间,清除缓存del "patriarch:parentAlertNoParentId:1147388_*"
UPDATE edu_parent_alert_log SET `status`=0,lastTime=FROM_UNIXTIME(UNIX_TIMESTAMP(lastTime)-2592000-10000) WHERE parentId=1147388
UPDATE edu_parent_alert_log SET `status`=0,lastTime=Date_sub(lastTime,interval 1 month ) WHERE parentId=1147388
UPDATE edu_parent_alert_log SET `status`=0,lastTime=Date_sub(lastTime,interval 31 day ) WHERE parentId=1147388
1.测试锦标赛领取接口
列表数据接口耗时
INSERT INTO match_prize (NAME,Type,MarketPrice)VALUES ('实物1','1','100');
SELECT COUNT(PrizeID) FROM match_prize;或者SELECT COUNT(*) FROM match_prize;
INSERT INTO match_user_prize (Userid,prizeid)VALUES ('1329447','16');
SELECT COUNT(*) FROM match_user_prize;
2.SELECT * FROM `match_match` WHERE Type=2;
3.测试全国锦标赛,涉及到的数据表 base_user,`match_template` (比赛模板),match_match
要修改年级
查询match_match年级,更新base_user年级为一致
-----------------------------------------------------------------------------------------------全国锦标赛首页列表接口
2.SELECT Level,MatchID FROM `match_match` WHERE Type in(2,3);
//`Type` tinyint(4) NOT NULL DEFAULT '1' COMMENT '比赛类型 1:部落锦标赛 2:胜利锦标赛 3:排名锦标赛',
3.两表关联
SELECT * FROM match_template AS a INNER JOIN match_match AS b on a.MatchTemplateID=b.MatchTemplateID AND a.BeginDate<="2017-03-26 00:00:0" AND a.EndDate>="2017-03-26 00:00:00";
4.查询学生表,设置参赛年级为1年级
SELECT * FROM `base_user` WHERE UserID=1329447;
UPDATE base_user SET GradePart=1;
用户锦标赛信息查询我的id下的胜利、失败次数
SELECT * FROM `match_student` WHERE StudentID=1329447 AND MatchID IN (447,463);
胜利需要赢的总pk次数(本次设置12次)
SELECT WinRequireTimes FROM match_template AS a INNER JOIN match_match AS b on a.MatchTemplateID=b.MatchTemplateID AND a.BeginDate<="2017-03-26 00:00:0" AND a.EndDate>="2017-03-26 00:00:00";
全国锦标赛包括胜利挑战赛(match_match和match_template)和排名挑战赛(match_match)
胜利赛数据
SELECT * FROM match_template AS a,match_match AS b WHERE a.MatchTemplateID=b.MatchTemplateID;
胜利赛
SELECT * FROM `match_student` WHERE StudentID=1329447 AND MatchType=2;
matchTplId对应数据库MatchTemplateID
全国胜利挑战赛子列表
接口match_tpl_id对应数据库里的MatchTemplateID
SELECT a.MatchID,a.MatchTemplateID FROM match_match AS a INNER JOIN match_template AS b on a.MatchTemplateID=b.MatchTemplateID;
内连隐式查询
SELECT a.MatchID FROM match_match AS a,match_template AS b WHERE a.MatchTemplateID=b.MatchTemplateID;
SELECT * FROM match_match as A INNER JOIN match_vls_result as B ON A.MatchID=B.MatchID AND A.MatchID in(10016,9979,9998)
SELECT sum(AnswerScore) FROM match_vls_result WHERE MatchID=10082
全国锦标赛排名赛详情涉及到的表
Match_match和 match_student
SELECT MatchStudentID FROM `match_student` WHERE StudentID=1329447;
SELECT * FROM match_match AS a,match_student AS b WHERE a.MatchID=b.MatchID AND a.Type=3 AND b.StudentID=1329447;
//type=3排名赛,根据studentid找到matchstudentid
全国锦标赛胜利赛详情涉及到的表3张
SELECT * FROM `match_student` WHERE StudentID=1329447;
SELECT * FROM match_match WHERE MatchID=447;
SELECT * FROM match_template WHERE MatchTemplateID=1;
SELECT * FROM match_student a,match_match b,match_template c WHERE a.MatchID=b.MatchID AND b.MatchTemplateID=c.MatchTemplateID AND a.MatchID=447 AND a.StudentID=1329447;
全国锦标赛胜利详情页
给用户充值钻石
SELECT * FROM `base_student` WHERE StudentID=1068708;
UPDATE base_student SET DiamondCount=100 WHERE StudentID=1068708;
//毫秒转换为分钟
SELECT sum(TimeUsed/1000/60) FROM match_vls_record WHERE MatchID=9979 AND StudentID=1573051
//全国排名赛nationrank
drop procedure if exists `nationrank`;
delimiter //
create procedure nationrank()
begin
declare num int;
SET num=1;
while num < 1002 do
INSERT INTO match_student (MatchID,MatchType,Score,IsJoin,PkTimes) VALUES (10145,3,67,1,1);
SET num=num+1;
end while;
end
//
call nationrank();
SELECT * FROM match_student WHERE MatchID=10145;
UPDATE match_student SET studentID=MatchStudentID-303414+10001 where matchID=10145
UPDATE match_student SET score=10 where matchID=10145
UPDATE match_match SET EndDate="2017-06-10 16:00:00",StopTime="16:00:00",LastStopTime="2017-06-10 16:00:00" WHERE MatchID=10103;
SELECT * FROM match_match as a INNER JOIN match_student as b on a.MatchID=b.MatchID AND a.MatchID=9995;
//GROUP_CONCAT函数返回一个字符串结果,该结果由分组中的值连接组合而成
SELECT group_concat(MatchStudentID) FROM match_student WHERE MatchID=9995
UPDATE match_student SET score=50,IsJoin=1,PkTimes=1 WHERE MatchStudentID in(218398,218476,218531,218875)
//全国单词测验造数据1000条,非补测数据。是否补测,0否,1是,测验专用
drop procedure if exists `nationtest`;
delimiter //
create procedure nationtest()
begin
declare num int;
SET num=1;
while num < 1000 do
INSERT INTO match_student (MatchID,MatchType,Score,IsJoin,PkTimes,TimeUsed,RewardCoin,RewardStatus) VALUES (10163,7,61,1,19,31781,9,1); SET num=num+1;
end while;
end
//
call nationtest();
//全国单词测验造数据1000条,补测数据。是否补测,0否,1是,测验专用
drop procedure if exists `nationtestsupply`;
delimiter //
create procedure nationtestsupply()
begin
declare num int;
SET num=1;
while num < 10 do
INSERT INTO match_student (MatchID,MatchType,Score,IsJoin,PkTimes,TimeUsed,RewardCoin,RewardStatus,IsSupply) VALUES (10103,7,87,1,6,31781,9,1,1);
SET num=num+1;
end while;
end
//
call nationrank();
//插入1000条数据,studentID去重,但需要studentID不为空且从base_student取1000。需要设置为自增id+1,不能设置studentID+1
UPDATE match_student SET studentID=MatchStudentID-306240+10151,PkTimes=19 where matchID=10163
UPDATE match_student SET Score=88 WHERE
SELECT StudentID FROM match_student where MatchID=10103;
SELECT COUNT(*) FROM match_student WHERE MatchID=10163;
SELECT * FROM match_student WHERE MatchID=10163 AND IsSupply=1 AND Score=100;
DELETE FROM match_student WHERE MatchID=10151 LIMIT 1000;
SELECT * FROM match_match WHERE MatchID=10163 ;
UPDATE match_match SET EndDate="2017-06-10 16:00:00",StopTime="16:00:00",LastStopTime="2017-06-10 16:00:00" WHERE MatchID=10103;
UPDATE match_match SET EndDate="2017-06-10 22:00:00",StopTime="22:00:00",LastStopTime="2017-06-10 22:00:00" WHERE MatchID=10103;
SELECT * FROM base_student LIMIT 10;
drop procedure if exists `nationtestB`;
delimiter //暂时无用
create procedure nationtestB()
begin
declare num int;
SET num=1;
while num < 10 do
UPDATE match_student SET StudentID=(SELECT studentID FROM (SELECT studentID,count(*) FROM base_student order by studentID desc limit num) as A) where MatchStudentID=
(SELECT matchStudentID FROM (SELECT matchStudentID,count(*) FROM match_student where matchID=10015 order by matchStudentID desc limit num) as B);
SET num=num+1;
end while;
end//
call nationtestB();
UPDATE match_student SET studentID=102094 where matchID=9995
SELECT distinct StudentID FROM match_student where matchID=10015;matchStudentID in (300142,300141,300140,300139);(SELECT matchStudentID FROM match_student where matchID=10015 order by matchStudentID desc limit 10);
//单词列表为非视听说题型 /match/vls-train/is-vls-train? isVls 1:视听说 0:非视听说
SELECT DISTINCT QuestionDim,wordID FROM word_question WHERE wordID NOT in ( SELECT DISTINCT wordID from word_question WHERE QuestionDim in (10,11,12,13) )
//怪物血量除以100
update monster_monster set InitHp=InitHp/100
update monster_monster set InitHp=InitHp*10
update monster_monster set InitHp=InitHp/25
select InitHp from monster_monster
//如果用GROUP_CONCAT函数,读取的StudentID为字符串,所以取得是第一个
SELECT * from base_user WHERE UserID in (SELECT GROUP_CONCAT(StudentID) from base_student_class WHERE ClassID=54629)
//单独的子查询是可以的,如果是使用group_concat 的话,得有group by才能运行处结果。若是没有使用group by时使用聚合函数,相当于把所有的行都归于一组来进行处理。除非特殊说明。一般使用聚合函数会忽略掉NULL
update task_template set Title='在班级锦标赛中获得1场胜利' where TemplateID=4 limit 1;
update带limit是安全做法,可以限制最大更新数据行数。不过这种写法有问题的
CREATE TABLE a
(col1 int,
col2 int);
CREATE TABLE b
(col1 int,
col2 int);
INSERT INTO a VALUES (1,1),(2,2);
INSERT INTO b VALUES (2,2);
SELECT * FROM a WHERE EXISTS (select 1 FROM b WHERE a.col2=b.col2);
https://zhidao.baidu.com/question/314655739.html
linux下的mysql就能查询了
DROP TABLE a;
CREATE TABLE a
(col BLOB);
INSERT INTO a VALUES ('123'),(123);
mysql> select * from a;
+------+
| col |
+------+
| 123 |
| 123 |
+------+
2 rows in set (0.00 sec)
=============================================================================
兑吧平台
UPDATE `base_student` SET CoinCount=90000 WHERE StudentID=1364361;
SELECT userID from base_user WHERE Mobile=18500214570 AND UserType=0
UPDATE base_student SET EnExperience=32600 WHERE StudentID in (1364361);
SELECT StudentID,ErrorWordCount,EnExperience,EnLevel FROM base_student WHERE StudentID in (1329654,1329447);
=============================================================================
学生端1.8.0&老师端1.5.0
SELECT * FROM base_class as a
INNER JOIN base_user as b ON b.UserID=a.TeacherID AND b.Mobile =13031076451
AND b.UserType=1
//`Type`:测验类型:1单词,2听力,3口语,4阅读
SELECT * from exam_exam WHERE ClassID=49076 AND Type=1
SELECT * from exam_student WHERE ClassID=49076 AND Type=1
SELECT sum(RightRate)/count(*) from exam_class_answer_statis WHERE ClassID=49076 AND Type=1
SELECT * from exam_class_answer_statis WHERE ClassID=49076 AND Type=1
//老师端正确率=学生正确率总和/人数
SELECT sum(RightRate)/count(*) from exam_class_answer_statis WHERE ClassID=49076 AND Type=2 AND ExamID=366
SELECT sum(RightRate)/count(*) from exam_class_answer_statis WHERE ClassID=49076 AND Type=3
SELECT * from exam_class_answer_statis WHERE ClassID=49076 AND Type=3
SELECT * from exam_student WHERE ClassID=49076 AND Type=3 AND ExamID=370
SELECT * from exa WHERE ClassID=49076 AND Type=3 AND ExamID=370
SELECT * from exam_answer WHERE QuestionID=SubQuestionID
SELECT * from exam_student WHERE ClassID=49076 AND Type=3
SELECT * from exam_exam WHERE ClassID=39174
SELECT * from exam_exam WHERE ClassID=39174
SELECT * from word_quespackage WHERE QPID=587
SQL审核
alter table exam_exam add Switch int unsigned not null default 0 comment '开关,.0结算答案不可见,1结算答案可见';
update task_template set Title='在班级锦标赛中获得1场胜利' where TemplateID=4 limit 1;
update带limit是安全做法,可以限制最大更新数据行数。不过这种写法有问题的
=============================================================================
学生端1.6.0
//学生打怪表
select * from monster_class_monster WHERE ClassMonsterID=1337
select * from monster_class_monster WHERE ClassID=48606
UPDATE monster_class_monster SET ClassID=0 WHERE ClassID=48606
select * from monster_student_monster WHERE ClassMonsterID=1301 GROUP BY StudentID ORDER BY Damage DESC
select * from monster_student_monster WHERE StudentID=1743086 ORDER BY AddTime Desc
UPDATE monster_student_monster SET ClassID=0 WHERE ClassID=48606
select * from monster_student_monster ORDER BY AddTime Desc
DELETE * from monster_student_monster WHERE StudentID in (1743043,1743040)
select * from monster_student_monster WHERE StudentID in (1743045,1743044)
select sum(Damage) from monster_student_monster WHERE ClassMonsterID=1094
select * from monster_monster WHERE MonsterID=1
select * from monster_class_monster as a INNER join monster_monster
as b ON a.MonsterID=b.MonsterID AND a.ClassMonsterID=1143
//伤害榜pk记录,根据pkResult的"sourceID": 766去monster_student_pk_record查询,生成了结算数据
select * from monster_student_pk_record WHERE recordID=766
select * from monster_student_pk_record WHERE StudentID=1101752 AND MonsterStatus=1
select * from monster_student_pk_record WHERE StudentID=1252141 ORDER BY AddTime desc
select sum(Damage) from monster_student_pk_record WHERE ClassMonsterID=1076
select StudentID,sum(Damage) from monster_student_pk_record WHERE ClassMonsterID=1194 GROUP BY StudentID ORDER BY SUM(Damage)
select * from monster_student_pk_record WHERE ClassMonsterID=1152 GROUP BY StudentID ORDER BY Damage desc
select * from monster_student_pk_record GROUP BY StudentID ORDER BY ClassMonsterID desc
select * from base_class WHERE ClassID=18072
select * from base_student_class WHERE ClassID=18072
select count(*) from base_student_class WHERE ClassID=18072
select * from gym_user_skill WHERE StudentID=1573049
select * from base_user WHERE Mobile=15321895622 AND UserType=0
select * from match_class_message WHERE `Content` LIKE "%1080966%"
//修改用户团队打怪领奖记录
select * from monster_student_monster WHERE ClassMonsterID=1163 ORDER BY Damage DESC
select * from monster_student_monster ORDER BY AddTime DESC
select * from monster_student_monster WHERE StudentID=1080922
select * from monster_student_monster WHERE StudentID=1743035
select * from monster_student_monster WHERE Reward not in("[]","") ;
select * from monster_student_monster WHERE ClassMonsterID=1194 ORDER BY StudentMonsterID DESC
select * from monster_student_Monster where studentid in (select studentID from monster_student_monster where classMonsterID=1194) and monsterID=2 and weekNo='201729' order BY studentID asc
//给用户充值钻石
select * from`base_student` WHERE StudentID=1743038;
select * from base_user WHERE UserID=1743038
select * from base_user WHERE Mobile=18579051698 AND UserType=0
UPDATE base_student SET DiamondCount=100000 WHERE StudentID=1743105;
//怪物血量除以100
update monster_monster set InitHp=InitHp/100
update monster_monster set InitHp=InitHp*10
update monster_monster set InitHp=InitHp/25
select InitHp from monster_monster
SELECT * from match_match ORDER BY CreateTime DESC LIMIT 10