Logstash-Output-Webhdfs

无奈,有时候为了解决问题而找答案,解决后却忘了作笔记..这次狠狠给自己个教训..
[WARN ][logstash.outputs.webhdfs ] Failed to flush outgoing items {:outgoing_count=>500, :exception=>”WebHDFS::ServerError”, :backtrace=>[“/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:351:in request'", "/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:270:inoperate_requests’”, “/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/webhdfs-0.8.0/lib/webhdfs/client_v1.rb:73:in create'", "/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:211:inwrite_data’”, “/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:195:in flush'", "org/jruby/RubyHash.java:1342:ineach’”, “/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:183:in flush'", "/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:219:inbuffer_flush’”, “org/jruby/RubyHash.java:1342:in each'", "/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:216:inbuffer_flush’”, “/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:193:in buffer_flush'", "/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/buffer.rb:159:inbuffer_receive’”, “/opt/logstash-5.4.1/vendor/bundle/jruby/1.9/gems/logstash-output-webhdfs-3.0.2/lib/logstash/outputs/webhdfs.rb:166:in receive'", "/opt/logstash-5.4.1/logstash-core/lib/logstash/outputs/base.rb:92:inmulti_receive’”, “org/jruby/RubyArray.java:1613:in each'", "/opt/logstash-5.4.1/logstash-core/lib/logstash/outputs/base.rb:92:inmulti_receive’”, “/opt/logstash-5.4.1/logstash-core/lib/logstash/output_delegator_strategies/legacy.rb:22:in multi_receive'", "/opt/logstash-5.4.1/logstash-core/lib/logstash/output_delegator.rb:47:inmulti_receive’”, “/opt/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:407:in output_batch'", "org/jruby/RubyHash.java:1342:ineach’”, “/opt/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:406:in output_batch'", "/opt/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:352:inworker_loop’”, “/opt/logstash-5.4.1/logstash-core/lib/logstash/pipeline.rb:317:in `start_workers’”]}

想半天,hdfs不可能有问题的,线上一直跑着呢,原来webhdfs的配置中user是不必须填的,倒腾半天,没结果查下网上,终于找到了答案,解决了,随便自己也作下笔记..

webhdfs
user:必须填写启动hdfs的用户名
必须在logstash所在的服务器上的/etc/hosts配置hdfs集群节点路由列表
分享借鉴地址:http://stackmirror.cn/page/s1b6a4mbbvas