Faacbook’s arctiehcture(转)

FromvariousreadingsandconversationsIhad,myunderstandingofFacebook’scurrentarchitectureis:

*Webfront-endwritteninPHP.Facebook’sHipHop[1]thenconvertsittoC++andcompilesitusingg++,thusprovidingahighperformancetemplatingandWeblogicexecutionlayer
*BusinesslogicisexposedasservicesusingThrift[2].SomeoftheseservicesareimplementedinPHP,C++orJavadependingonservicerequirements(someotherlanguagesareprobablyused…)
*ServicesimplementedinJavadon’tuseanyusualenterpriseapplicationserverbutratheruseFacebook’scustomapplicationserver.Atfirstthiscanlookaswheelreinventedbutastheseservicesareexposedandconsumedonly(ormostly)usingThrift,theoverheadofTomcat,orevenJettywasprobablytoohighwithnosignificantaddedvaluefortheirneed.
*PersistenceisdoneusingMySQL,Memcached[3],Facebook’sCassandra[4],Hadoop’sHBase[5].MemcachedisusedasacacheforMySQLaswellasageneralpurposecache.FacebookengineersadmitthattheiruseofCassandraiscurrentlydecreasingastheynowpreferHBaseforitssimplerconsistencymodelanditsMapReduceability.
*OfflineprocessingisdoneusingHadoopandHive
*Datasuchaslogging,clicksandfeedstransitusingScribe[6]andareaggregatingandstoredinHDFSusingScribe-HDFS[7],thusallowingextendedanalysisusingMapReduce
*BigPipe[8]istheircustomtechnologytoacceleratepagerenderingusingapipelininglogic
*VarnishCache[9]isusedforHTTPproxying.They’veprefereditforitshighperformanceandefficiency[10].
*ThestorageofthebillionsofphotospostedbytheusersishandledbyHaystack,anad-hocstoragesolutiondevelopedbyFacebookwhichbringslowleveloptimizationsandappend-onlywrites[11].
*FacebookMessagesisusingitsownarchitecturewhichisnotablybasedoninfrastructureshardinganddynamicclustermanagement.Businesslogicandpersistenceisencapsulatedinso-called‘Cell’.EachCellhandlesapartofusers;newCellscanbeaddedaspopularitygrows[12].PersistenceisachievedusingHBase[13].
*FacebookMessages’searchengineisbuiltwithaninvertedindexstoredinHBase[14]
*FacebookSearchEngine’simplementationdetailsareunknownasfarasIknow
*Thetypeaheadsearchusesacustomstorageandretrievallogic[15]
*ChatisbasedonanEpollserverdevelopedinErlangandaccessedusingThrift[16]

Abouttheresourcesprovisionedforeachofthesecomponents,someinformationandnumbersareknown:

*Facebookisestimatedtoownmorethan60,000servers[17].TheirrecentdatacenterinPrineville,Oregonisbasedonentirelyself-designedhardware[18]thatwasrecentlyunveiledasOpenComputeProject[19].
*300TBofdataisstoredinMemcachedprocesses[20]
*TheirHadoopandHiveclusterismadeof3000serverswith8cores,32GBRAM,12TBdisksthatisatotalof24kcores,96TBRAMand36PBdisks[20]
*100billionhitsperday,50billionphotos,3trillionobjectscached,130TBoflogsperdayasofjuly2010[21]

[1]HipHopforPHP:http://developers.facebook.com/blog/post/358
[2]Thrift:http://thrift.apache.org/
[3]Memcached:http://memcached.org/
[4]Cassandra:http://cassandra.apache.org/
[5]HBase:http://hbase.apache.org/
[6]Scribe:https://github.com/facebook/scribe
[7]Scribe-HDFS:http://hadoopblog.blogspot.com/2009/06/hdfs-scribe-integration.html
[8]BigPipe:http://www.facebook.com/notes/facebook-engineering/bigpipe-pipelining-web-pages-for-high-performance/389414033919
[9]VarnishCache:http://www.varnish-cache.org/
[10]FacebookgoesforVarnish:http://www.varnish-software.com/customers/facebook
[11]Needleinahaystack:efficientstorageofbillionsofphotos:http://www.facebook.com/note.php?note_id=76191543919
[12]ScalingtheMessagesApplicationBackEnd:http://www.facebook.com/note.php?note_id=10150148835363920
[13]TheUnderlyingTechnologyofMessages:https://www.facebook.com/note.php?note_id=454991608919
[14]TheUnderlyingTechnologyofMessagesTechTalk:http://www.facebook.com/video/video.php?v=690851516105
[15]Facebook’stypeaheadsearcharchitecture:http://www.facebook.com/video/video.php?v=432864835468
[16]FacebookChat:http://www.facebook.com/note.php?note_id=14218138919
[17]WhohasthemostWebServers?:http://www.datacenterknowledge.com/archives/2009/05/14/whos-got-the-most-web-servers/
[18]BuildingEfficientDataCenterswiththeOpenComputeProject:http://www.facebook.com/note.php?note_id=10150144039563920
[19]OpenComputeProject:http://opencompute.org/
[20]Facebook’sarchitecturepresentationatDevoxx2010:http://www.devoxx.com
[21]ScalingFacebookto500millionsusersandbeyond:http://www.facebook.com/note.php?note_id=409881258919