高并发update时的几种处理方式

看了德哥的 https://yq.aliyun.com/articles/3010 这篇文章，觉得很实用，自己实践了一把，记录一下。

数据准备

peiybdb=# create table tmp_t0(c0 int8, c1 int8);
peiybdb=# create index idx_tmp_t0_c1 on tmp_t0(c1);
peiybdb=# insert into tmp_t0 select id,id from generate_series(1,1000000) as id;

update,select for update

最基本的处理方式，等待时间较长，并发处理能力低

peiybdb=# CREATE OR REPLACE FUNCTION public.f1(ip_id integer)  
 RETURNS void  
 LANGUAGE plpgsql  
AS $function$ 
declare 
begin  
  update tmp_t0 set c0=c0+1 where c1=ip_id; 
  exception when others then  
  return; 
end; 
$function$;


$ vi /tmp/f1.sql
\set id random(999,999)
select f1(:id);

select for update nowait

nowait 方式，有较高的处理能力

peiybdb=# CREATE OR REPLACE FUNCTION public.f2(ip_id integer)  
 RETURNS void  
 LANGUAGE plpgsql  
AS $function$ 
declare 
begin 
  perform 1 from tmp_t0 where c1=ip_id for update nowait; 
  update tmp_t0 set c0=c0+1 where c1=ip_id; 
  exception when others then  
  return; 
end; 
$function$;


$ vi /tmp/f2.sql
\set id random(999,999)
select f2(:id);

select for update skip locked

skip locked 方式，有较高的处理能力

peiybdb=# CREATE OR REPLACE FUNCTION public.f3(ip_id integer)  
 RETURNS void  
 LANGUAGE plpgsql  
AS $function$ 
declare 
begin 
  update tmp_t0 set c0=c0+1 where ctid = any (array(select ctid from tmp_t0 t0 where t0.c1=ip_id for update skip locked)); 
  exception when others then  
  return; 
end; 
$function$;


$ vi /tmp/f3.sql
\set id random(999,999)
select f3(:id);

advisory lock

advisory lock，很高的处理能力
使用 advisory lock 一定要针对最少量的行数，最好使用主键处理单条记录。

peiybdb=# CREATE OR REPLACE FUNCTION public.f4(ip_id integer)  
 RETURNS void  
 LANGUAGE plpgsql  
AS $function$ 
declare 
begin 
  if (select pg_try_advisory_xact_lock(ip_id) ) = true then
    update tmp_t0 set c0=c0+1 where c1=ip_id ; 
  end if;
  exception when others then  
  return; 
end; 
$function$;


$ vi /tmp/f4.sql
\set id random(999,999)
select f4(:id);

pgbench 测试

直接 update 结果如下:

$ pgbench -n -r -P 1 -f /tmp/f1.sql -c 20 -j 4 -T 60 peiybdb

transaction type: /tmp/f1.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 4
duration: 60 s
number of transactions actually processed: 39163
latency average = 30.640 ms
latency stddev = 33.755 ms
tps = 652.293068 (including connections establishing)
tps = 652.334596 (excluding connections establishing)
script statistics:
 - statement latencies in milliseconds:
         0.002  \set id random(999,999)
        30.638  select f1(:id);

select for update nowait 结果如下:

$ pgbench -n -r -P 1 -f /tmp/f2.sql -c 20 -j 4 -T 60 peiybdb

transaction type: /tmp/f2.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 4
duration: 60 s
number of transactions actually processed: 1536810
latency average = 0.780 ms
latency stddev = 1.845 ms
tps = 25610.329868 (including connections establishing)
tps = 25613.687578 (excluding connections establishing)
script statistics:
 - statement latencies in milliseconds:
         0.001  \set id random(999,999)
         0.779  select f2(:id);
		 

select for update skip locked 结果如下:

$ pgbench -n -r -P 1 -f /tmp/f3.sql -c 20 -j 4 -T 60 peiybdb

transaction type: /tmp/f3.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 4
duration: 60 s
number of transactions actually processed: 1505587
latency average = 0.796 ms
latency stddev = 1.483 ms
tps = 25089.353264 (including connections establishing)
tps = 25090.922905 (excluding connections establishing)
script statistics:
 - statement latencies in milliseconds:
         0.001  \set id random(999,999)
         0.796  select f3(:id);

advisory lock 结果如下:

$ pgbench -n -r -P 1 -f /tmp/f4.sql -c 20 -j 4 -T 60 peiybdb

transaction type: /tmp/f4.sql
scaling factor: 1
query mode: simple
number of clients: 20
number of threads: 4
duration: 60 s
number of transactions actually processed: 2112088
latency average = 0.567 ms
latency stddev = 1.430 ms
tps = 35177.565856 (including connections establishing)
tps = 35181.744602 (excluding connections establishing)
script statistics:
 - statement latencies in milliseconds:
         0.001  \set id random(999,999)
         0.567  select f4(:id);
		 

-M, --protocol=simple|extended|prepared
protocol for submitting queries (default: simple)

结论

advisory lock 处理能力最高，nowait 和 skip locked 差别不是特别大，直接 for update 最差。

而且直接 for update 还可能导致数据库快速达到最大连接数，或者应用的连接池爆满的情况，是最简单粗暴的操作。

参考：
https://yq.aliyun.com/articles/3010
https://github.com/digoal/blog/blob/master/201801/20180105_03.md
http://postgres.cn/docs/9.6/explicit-locking.html#ADVISORY-LOCKS