Facebooks BigPipe Done in Java
BigPipe is a way of thinking for web pages. It introduces the concept of pagelets, small parts of the website. BigPipe was implemented at Facebook:
BigPipe is a fundamental redesign of the dynamic web page serving system. The general idea is to decompose web pages into small chunks called pagelets, and pipeline them through several execution stages inside web servers and browsers. This is similar to the pipelining performed by most modern microprocessors: multiple instructions are pipelined through different execution units of the processor to achieve the best performance.
BigPipe renders the structure of a webpage and then adds the content via JavaScript, a kind of inlined AJAX. The result is that users see content earlier and progressivly. Just reload a Facebook page and see how different parts are loaded. This reduces latency and especially changes the perception of users:They think the website is faster. A side effect of splitting the page into pagelets is how each pagelet can be rendered in parallel with asynchronous IO on the server or with an IO thread pool.
With threads, message piping or a worker model very long running service calls do not stop the page from loading, it will only prevent showing the pagelet that is slow. Each thread can have a timeout and you can kill long running or blocking service calls – a SLA for page calls.
Sam Pullaras mustache.java follows the same principles, breaking rendering into parts each with it’s own thread to reduce latency.
I’ve implemented a proof of concept of BigPipe in Java (should run as-is in every servlet container):
public
class
BigPipeServlet
extends
HttpServlet
{
private
static
ExecutorService
executor
=
Executors
.
newFixedThreadPool
(
500
,
new
ThreadFactory
()
{
public
Thread
newThread
(
Runnable
r
)
{
Thread
t
=
new
Thread
(
r
);
t
.
setName
(
"Service Thread "
+
t
.
getId
());
t
.
setDaemon
(
true
);
return
t
;
}
});
protected
void
service
(
HttpServletRequest
req
,
final
HttpServletResponse
resp
)
throws
ServletException
,
IOException
{
final
PrintWriter
writer
=
resp
.
getWriter
();
String
doctype
=
"<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1.0 Strict//EN\"\n"
+
" \"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd\">"
;
String
head
=
"<html xmlns=\"http://www.w3.org/1999/xhtml\" xml:lang=\"de\" lang=\"de\"> \n"
+
"<head> \n"
+
"<meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" /> \n"
+
"<meta http-equiv=\"Content-language\" content=\"de\" />\n"
;
writer
.
write
(
doctype
);
writer
.
write
(
head
);
writer
.
write
(
"<script type=\"text/javascript\">function arrived(id,text) { var b=document.getElementById(id); b.innerHTML = text; }</script>"
);
writer
.
write
(
"</HEAD><BODY><div>Progressive Loading"
);
content
(
writer
,
"content1"
,
"content2"
,
"content3"
,
"content4"
,
"content5"
,
"content6"
);
writer
.
write
(
"</div>\n"
);
final
Random
random
=
new
Random
();
List
<
Callable
<
Boolean
>>
tasks
=
new
ArrayList
<
Callable
<
Boolean
>>();
for
(
int
i
=
0
;
i
<
6
;
i
++)
{
final
int
id
=
i
+
1
;
tasks
.
add
(
new
Callable
<
Boolean
>()
{
public
Boolean
call
()
{
try
{
// One service call is nasty and takes 50sec
if
(
id
==
3
)
{
Thread
.
sleep
(
50000
);
}
else
{
Thread
.
sleep
(
random
.
nextInt
(
2000
));
}
pagelet
(
writer
,
"content"
+
id
,
"Wohooo"
+
id
);
}
catch
(
InterruptedException
e
)
{
return
false
;
}
return
true
;
}
});
}
try
{
executor
.
invokeAll
(
tasks
,
1500
,
TimeUnit
.
MILLISECONDS
);
}
catch
(
InterruptedException
ignored
)
{
// ignored
}
writer
.
write
(
"</BODY></HTML>"
);
}
private
void
content
(
PrintWriter
writer
,
String
...
contentIds
)
{
for
(
String
id
:
contentIds
)
{
writer
.
write
(
"<div id=\""
+
id
+
"\">-</div>\n"
);
}
}
private
void
pagelet
(
PrintWriter
writer
,
String
id
,
String
content
)
{
if
(
writer
.
checkError
())
return
;
writer
.
write
(
"<script>"
+
"arrived(\""
+
id
+
"\", \""
+
content
+
"\");"
+
"</script>\n"
);
writer
.
flush
();
}
}
This will result in the following HTML code – see how content2 is rendered before content1. Due to threads the order in which content arrives is non-deterministic.
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html
xmlns=
"http://www.w3.org/1999/xhtml"
xml:lang=
"de"
lang=
"de"
id=
"facebook"
class=
" no_js"
>
<head>
<meta
http-equiv=
"Content-type"
content=
"text/html; charset=utf-8"
/>
<meta
http-equiv=
"Content-language"
content=
"de"
/>
<script
type=
"text/javascript"
>
function
arrivedHtml
(
id
,
text
)
{
var
b
=
document
.
getElementById
(
id
);
b
.
innerHTML
=
text
;
}
</script></HEAD><BODY>
<div>Progressive Loading
<div
id=
"content1"
>-
</div>
<div
id=
"content2"
>-
</div>
</div>
<script>
arrivedHtml
(
"content2"
,
"Wohoo"
);
</script>
<script>
arrivedHtml
(
"content1"
,
"Wohoo"
);
</script>
</BODY></HTML>
Some problems I’ve encountered:
- You need the correct content type, content encoding and doc versiom, otherwise the page will not render progressivly
- Your framework needs to enable flushing the output at certain points, so content is pushed to the browser
I would like to have some framework support and I’m taking a look into how to do this in Play.
You can leave a Reply here. Of course, you should follow me on twitter here.