Sidekiq concurrency 设置和一些注意事项

使用 sidekiq 作为异步队列在耗时任务场景中极大地提高了用户体验。一个场景就是数字货币的汇率以及法币的外汇需要去第三方接口获取数据，如果对数据实时性要求不是很高的话可以放到异步队列中定时周期请求第三方的接口，然后把数据放到缓存中，此举极大地提高了系统的用户体验。
一个 sidekiq 进程默认开启 25 个线程，不过我们也可以在配置中更改并发数。有两种方式，第一种是启动队列时命令带上 -c 10 参数。第二种时在配置文件中改。

// config/sidekiq.yml

:pidfile: ./tmp/pids/sidekiq.pid
:logfile: ./log/sidekiq.log
:concurrency: 10

另外不建议设置并发数超过 50 ，还需要注意一点的时，ActiveRecord 在 config/database.yml 中有一个数据库连接池，此举就是为了高并发时系统不会因为数据库频繁操作降低了系统处理业务的性能。但是 sidekiq 的 concurrency 要低于 config/database.yml 中 pool 设置的值。
由于一开始不知道这点， pool 值低于 concurrency ，导致线上 sidekiq 日志报出了很多下面的错误，改好了之后再也不出现这样的错误日志了。

2016-11-25T04:50:43.013Z 32564 TID-afadw WARN: {"context":"Job raised exception","job":{"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GetRakeJob","queue":"get_rake","args":[{"job_class":"GetRakeJob","job_id":"4266206e-f55f-4401-9386-735e60b43044","queue_name":"get_rake","priority":null,"arguments":[],"locale":"zh-CN"}],"retry":true,"jid":"664246534cb9a8a06a7ecaca","created_at":1479985331.5119665,"enqueued_at":1480048630.4018834,"error_message":"could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use","error_class":"ActiveRecord::ConnectionTimeoutError","failed_at":1479985331.5130775,"retry_count":13,"retried_at":1480049443.0114233},"jobstr":"{"class":"ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper","wrapped":"GetRakeJob","queue":"get_rake","args":[{"job_class":"GetRakeJob","job_id":"4266206e-f55f-4401-9386-735e60b43044","queue_name":"get_rake","priority":null,"arguments":[],"locale":"zh-CN"}],"retry":true,"jid":"664246534cb9a8a06a7ecaca","created_at":1479985331.5119665,"enqueued_at":1480048630.4018834,"error_message":"Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)","error_class":"Mysql2::Error","failed_at":1479985331.5130775,"retry_count":12,"retried_at":1480026354.3648694}"}

推荐阅读

• sidekiq advanced options
• Better integration between Sidekiq thread count and Active Record pool size