为什么 open-uri 有时候是stringio 有时候返回 tmpfile?
为什么 open-uri 有时候是stringio 有时候返回 tmpfile?
我用ruby openuri 慢.
1.网络io文件大小未定,作为类库的设计者.
小文件太多,不好.IO性能差.
单个文件太大,如果都用memorybuffer,内存占用会非常的大.
设计者考虑2边的灵活性.
2.我实际使用为什么会慢.因为网络慢,丢包厉害.调整这个参数确实没什么用.
在irb中调用
require 'open-uri'
zipfile = open("https://www.baidu.com/img/bdlogo.png")
require 'open-uri'
# Don't allow downloaded files to be created as StringIO. Force a tempfile to be created.
OpenURI::Buffer.send :remove_const, 'StringMax' if OpenURI::Buffer.const_defined?('StringMax')
OpenURI::Buffer.const_set 'StringMax', 2000000
irb(main):010:0> zipfile = open("https://www.baidu.com/img/bdlogo.png")
=> #<Tempfile:/var/folders/ss/kfvmx1195cj_zz3zb3tg0ch00000gn/T/open-uri20150617-13110-yt0suh>
修改配置后测试
require 'open-uri'
# Don't allow downloaded files to be created as StringIO. Force a tempfile to be created.
OpenURI::Buffer.send :remove_const, 'StringMax' if OpenURI::Buffer.const_defined?('StringMax')
OpenURI::Buffer.const_set 'StringMax', 0
irb(main):004:0> zipfile = open("https://www.baidu.com/img/bdlogo.png")
=> #<StringIO:0x007fe2f9008b68 @base_uri=#<URI::HTTPS https://www.baidu.com/img/
参考:
http://stackoverflow.com/questions/10496874/why-does-openuri-treat-files-under-10kb-in-size-as-stringio
When one does network programming, you allocate a buffer of a reasonably large size and send and read units of data which will fit in the buffer. However, when dealing with files (or sometimes things called BLOBs) you cannot assume that the data will fit into your buffer. So, you need special handling for these large streams of data.
(Sometimes the units of data which fit into the buffer are called packets. However, packets are really a layer 4 thing, like frames are at layer 2. Since this is happening a layer 7, they might better be called messages.)
For replies larger than 10K, the open-uri library is setting up the extra overhead to write to a stream objects. When under the StringMax size, it just includes the string in the message, since it knows it can fit in the buffer.