HttpURLConnection OOM问题记录

使用HttpURLConnection 上传大文件，会出现内存溢出问题：

观察HttpURLConnection 源码：

@Overridepublic synchronized OutputStream getOutputStream() throws IOException {
    connecting = true;
    SocketPermission p = URLtoSocketPermission(this.url);

    if (p != null) {
        try {
            return AccessController.doPrivilegedWithCombiner(
                new PrivilegedExceptionAction<>() {
                    public OutputStream run() throws IOException {
                        return getOutputStream0();
                    }
                }, null, p            );
        } catch (PrivilegedActionException e) {
            throw (IOException) e.getException();
        }
    } else {
        return getOutputStream0();
    }
}

private synchronized OutputStream getOutputStream0() throws IOException {
    try {
        if (!doOutput) {
            throw new ProtocolException("cannot write to a URLConnection"                           + " if doOutput=false - call setDoOutput(true)");
        }

        if (method.equals("GET")) {
            method = "POST"; // Backward compatibility        }
        if ("TRACE".equals(method) && "http".equals(url.getProtocol())) {
            throw new ProtocolException("HTTP method TRACE" +
                                        " doesn't support output");
        }

        // if there's already an input stream open, throw an exception        if (inputStream != null) {
            throw new ProtocolException("Cannot write output after reading input.");
        }

        if (!checkReuseConnection())
            connect();

        boolean expectContinue = false;
        String expects = requests.findValue("Expect");
        if ("100-Continue".equalsIgnoreCase(expects) && streaming()) {
            http.setIgnoreContinue(false);
            expectContinue = true;
        }

        if (streaming() && strOutputStream == null) {
            writeRequests();
        }

        if (expectContinue) {
            expect100Continue();
        }
        ps = (PrintStream)http.getOutputStream();
        if (streaming()) {
            if (strOutputStream == null) {
                if (chunkLength != -1) { /* chunked */                     strOutputStream = new StreamingOutputStream(
                           new ChunkedOutputStream(ps, chunkLength), -1L);
                } else { /* must be fixed content length */                    long length = 0L;
                    if (fixedContentLengthLong != -1) {
                        length = fixedContentLengthLong;
                    } else if (fixedContentLength != -1) {
                        length = fixedContentLength;
                    }
                    strOutputStream = new StreamingOutputStream(ps, length);
                }
            }
            return strOutputStream;
        } else {
            if (poster == null) {
                poster = new PosterOutputStream();
            }
            return poster;
        }
    } catch (RuntimeException e) {
        disconnectInternal();
        throw e;
    } catch (ProtocolException e) {
        // Save the response code which may have been set while enforcing        // the 100-continue. disconnectInternal() forces it to -1        int i = responseCode;
        disconnectInternal();
        responseCode = i;
        throw e;
    } catch (IOException e) {
        disconnectInternal();
        throw e;
    }
}

public boolean streaming () {
    return (fixedContentLength != -1) || (fixedContentLengthLong != -1) ||
           (chunkLength != -1);
}

如上， 默认设置情况下streaming ()  为false。

package sun.net.www.http
public class PosterOutputStream extends ByteArrayOutputStream {
}

PosterOutputStream  默认为 ByteArrayOutputStream  子类

解决办法：

目标服务支持情况下，可以不使用HttpURLConnection的ByteArrayOutputStream缓存机制，直接将流提交到服务器上。如下函数设置：

httpConnection.setChunkedStreamingMode(0); // 或者设置自定义大小，0默认大小

    public void setChunkedStreamingMode (int chunklen) {
        if (connected) {
            throw new IllegalStateException ("Can't set streaming mode: already connected");
        }
        if (fixedContentLength != -1 || fixedContentLengthLong != -1) {
            throw new IllegalStateException ("Fixed length streaming mode set");
        }
        chunkLength = chunklen <=0? DEFAULT_CHUNK_SIZE : chunklen;
    }

遗憾的是，我上传的服务不支持这种模式。因此采用固定大小。

HttpURLConnection con = (HttpURLConnection)new URL("url").openConnection();
con.setFixedLengthStreamingMode(输出流的固定长度);

  /**
     * This method is used to enable streaming of a HTTP request body
     * without internal buffering, when the content length is known in
     * advance.
     * 

     * An exception will be thrown if the application
     * attempts to write more data than the indicated
     * content-length, or if the application closes the OutputStream
     * before writing the indicated amount.
     * 

     * When output streaming is enabled, authentication
     * and redirection cannot be handled automatically.
     * A HttpRetryException will be thrown when reading
     * the response if authentication or redirection are required.
     * This exception can be queried for the details of the error.
     * 

     * This method must be called before the URLConnection is connected.
     * 

     * NOTE: {@link #setFixedLengthStreamingMode(long)} is recommended
     * instead of this method as it allows larger content lengths to be set.
     *
     * @param   contentLength The number of bytes which will be written
     *          to the OutputStream.
     *
     * @throws  IllegalStateException if URLConnection is already connected
     *          or if a different streaming mode is already enabled.
     *
     * @throws  IllegalArgumentException if a content length less than
     *          zero is specified.
     *
     * @see     #setChunkedStreamingMode(int)
     * @since 1.5
     */
    public void setFixedLengthStreamingMode (int contentLength) {
        if (connected) {
            throw new IllegalStateException ("Already connected");
        }
        if (chunkLength != -1) {
            throw new IllegalStateException ("Chunked encoding streaming mode set");
        }
        if (contentLength < 0) {
            throw new IllegalArgumentException ("invalid content length");
        }
        fixedContentLength = contentLength;
    }

我是上传文件场景： 使用文件的大小作为长度

FileInputStream fileInputStream = new FileInputStream(uploadFileName);
long totalLength= fileInputStream.getChannel().size();
var boundary = "someboundary";
var temUploadUrl = "url path";
//
var url = new URL(temUploadUrl);
var connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("PUT");
connection.setRequestProperty("Content-Type", MediaType.MULTIPART_FORM_DATA_VALUE + "; boundary=" + boundary);

connection.setDoOutput(true);
// 设置 Content-Length
connection.setRequestProperty("Content-Length", String.valueOf(totalLength)); 
connection.setFixedLengthStreamingMode(totalLength);