zorka和zico的代码地址:https://github.com/jitlogic
由于zico是zorka的collecter端,所以在介绍zico之前首先说一下zorka和数据结构化存储和传输的部分。zorka抓取到数据后,会封装成一条TraceRecord,TraceRecord中包含本条trace的类,方法,调用链等信息,但是这些信息都是将对应的ID存在TraceRecord,而ID对应的详细信息会存在Symbol中,也就是说Symbol相当于一个数据字典。在传送和存储的之前会进行打TAG操作,已经来标记一条信息是Symbol的一个条目或者是TraceRecord中的一个条目。
首先,先试着理一下Zico工作的主要步骤:
ZicoServer通过socket建立连接,在run()方法中新建了ZicoServerConnector(继承自ZicoConnector),ZicoServerConnector中的核心部分是runCycle()函数。
//此方法在run方法中被调用 private void runCycle() throws IOException { ZicoPacket pkt = recv(); switch (pkt.getStatus()) { case ZICO_PING: { send(ZICO_PONG); break; } case ZICO_HELLO: { List
recv()函数是将通过socket读取的数据封装成一个ZicoPacket对象返回。在连接建立的初期,zico和zorka之间会发送ping-pong报文以及hello报文来测试建立的连接。ZicoPacket中的getData()方法返回一个byte数组,ZicoCommonUtil中的unpack()方法会将数据从byte数组中解包(unpack list of fressian-encoded objects from byte array)。unpack()函数执行的返回结果是List
只有ReceiverContext实现了ZicoDataProcessor接口,下面我们看一下ReceiverContext这个类。
@Override public synchronized void process(Object obj) throws IOException { if (hostStore.hasFlag(HostInfo.DELETED)) { log.info("Resetting connection for " + hostStore.getName() + " due to dirty SID map."); throw new ZicoException(ZicoPacket.ZICO_EOD, "Host has been deleted. Connection needs to be reset. Try again."); } if (hostStore.hasFlag(HostInfo.DISABLED)) { // Host store is disabled. Ignore incoming packets. return; } if (dirtySidMap) { log.info("Resetting connection for " + hostStore.getName() + " due to dirty SID map."); throw new ZicoException(ZicoPacket.ZICO_EOD, "Host was disabled, then enabled and SID map is dirty. Resetting connection."); } try { if (obj instanceof Symbol) { processSymbol((Symbol) obj); } else if (obj instanceof TraceRecord) { processTraceRecord((TraceRecord) obj); } else { if (obj != null) { log.warn("Unsupported object type:" + obj.getClass()); } else { log.warn("Attempted processing NULL object (?)"); } } } catch (Exception e) { log.error("Error processing trace record: ", e); } }
ReceiverContext中包含一个HostStore(Represents performance data store for a single agent)对象。
下面,我们看一下ReceiverContext中的processTraceRecord()方法。
private void processTraceRecord(TraceRecord rec) throws IOException { if (!hostStore.hasFlag(HostInfo.DISABLED)) { rec.traverse(this); visitedObjects.clear(); hostStore.processTraceRecord(rec); } else { log.debug("Dropping trace for inactive host: " + hostStore.getName()); } }
此方法中调用了TraceRecord的tracerse方法。
public void traverse(MetadataChecker checker) throws IOException { classId = checker.checkSymbol(classId, this); methodId = checker.checkSymbol(methodId, this); signatureId = checker.checkSymbol(signatureId, this); if (exception instanceof SymbolicException) { ((SymbolicException) exception).traverse(checker); } if (attrs != null) { MapnewAttrs = new LinkedHashMap (); for (Map.Entry e : attrs.entrySet()) { newAttrs.put(checker.checkSymbol(e.getKey(), this), e.getValue()); } attrs = newAttrs; } if (children != null) { for (TraceRecord child : children) { child.traverse(checker); } } if (marker != null && 0 != (flags & TRACE_BEGIN)) { marker.traverse(checker); } }
获取了三个id是****,读取了traceRecord及其孩子的attrs(Attributes grabbed at this method execution (by spy instrumentation engine))。
在processTraceRecord()方法中调用了HostStore类中的ProcessTraceRecord(TraceRecord obj)方法,在其中将TraceRecord(Represents trace information about single method execution,May contain references to information about calls from this method)包装成了TraceInfoRecord,TraceInfoRecord比TraceRecord多了四个属性:dataOffs,dataLen,indexOffs,indexLen。个人猜测用来控制块操作的。
在HostStore类的processTraceRecord()方法中将TraceRecord进行了更加详细的解析,将TraceRecord的对象分解为DataSore,IndexStore,infos等并且分别写入文件。
public void processTraceRecord(TraceRecord rec) throws IOException { TraceRecordStore traceDataStore = getTraceDataStore(); TraceRecordStore traceIndexStore = getTraceIndexStore(); BTreeMapinfos = getInfos(); Map tids = getTids(); if (traceDataStore == null || traceIndexStore == null || infos == null || tids == null || hasFlag(HostInfo.DISABLED|HostInfo.DELETED)) { throw new ZicoRuntimeException("Store " + getName() + " is closed and cannot accept records."); } TraceRecordStore.ChunkInfo dchunk = traceDataStore.write(rec); List tmp = rec.getChildren(); int numRecords = ZicoUtil.numRecords(rec); rec.setChildren(null); TraceRecordStore.ChunkInfo ichunk = traceIndexStore.write(rec); rec.setChildren(tmp); TraceInfoRecord tir = new TraceInfoRecord(rec,numRecords, dchunk.getOffset(), dchunk.getLength(), ichunk.getOffset(), ichunk.getLength(), rec.getAttrs() != null ? ZicoUtil.toIntArray(rec.getAttrs().keySet()) : null); checkAttrs(tir); infos.put(tir.getDataOffs(), tir); int traceId = tir.getTraceId(); if (!tids.containsKey(traceId)) { tids.put(traceId, symbolRegistry.symbolName(traceId)); } }
到此,总结一下数据从sokect读取到写入文件的过程,ZicoServerConnector中负责根据socket生成inputStream从socket中读取数据,此时读取的数据是fressian编码的byte数组,ZicoCommonUtil类的unpack()方法将这个byte数组解析为一个List
zico将收到的agent端的数据存储在zico/data文件下,目录结构如下图所示:
tdat中***存储的是host列表。
tidx:***(还不清楚)
host.properties中存储着host相关的信息(name,IP,group等等)
symbol.dat***
trace.db****
HostStore类代表一个agent的性能数据的存储(Represents performance data store for a single agent)。在HostStore类的构造函数中调用了open()函数,我们可以根据以下代码分析目录结构
public synchronized void open() { try { load(); if (symbolRegistry == null) { symbolRegistry = new PersistentSymbolRegistry( new File(ZicoUtil.ensureDir(rootPath), "symbols.dat")); } if (traceDataStore == null) { traceDataStore = new TraceRecordStore(config, this, "tdat", 1, this); } if (traceIndexStore == null) { traceIndexStore = new TraceRecordStore(config, this, "tidx", 4); } db = dbf.openDB(ZorkaUtil.path(rootPath, "traces.db"));
//private BTreeMap
infos = db.getTreeMap(DB_INFO_MAP);
//private Map
tids = db.getTreeMap(DB_TIDS_MAP);
//private BTreeMap
attrs = db.getTreeMap(DB_ATTR_MAP);
} catch (IOException e) { log.error("Cannot open host store " + name, e); } }
PersistentSymbolRegistry类继承自SymbolRegistry类,而SymbolRegistry类中包含了两个Map(symbolIds以及symbolNames)。PersistentSymbolRegistry类中的open()方法会从symbol.dat文件中读取symbol,并put进symbolIds和symbolNames。
TraceReocrdStore类在其构造函数中会新建一个RDSSotre(Raw Data Store (RDS) builds upon RAGZ stream classes)对象。在RSSsotre的open()方法中,会建立RAGZInputStream和RAGZOutputStream来读写RAGZSegment(represents a single in-memory segment)。RAGZSegment类有unpack()方法对数据进行解压,此处调用了java.util.zip包。
在zico-core包的pom文件中, 有以下语句
org.mapdb mapdb ${mapdb.version}
此处将mapdb导入了maven项目。对于trace.db文件使用了org.parboiled.BaseParser。 http://parboiled.org。这是一个轻量级no-SQL数据库
第二部分
zico使用了google guice IOC,在ProdZicoModule中有如下语句
@Override public void configure(Binder binder) { super.configure(binder); binder.bind(UserManager.class).asEagerSingleton(); binder.bind(UserContext.class).to(UserHttpContext.class); binder.bind(DBFactory.class).to(FileDBFactory.class); binder.bind(ZicoDataProcessorFactory.class).to(HostStoreManager.class); }
先分析最后一句话,ZicoDataProcessorFactory接口中只有一个get方法,此方法在HostStoreManager方法中实现如下
@Override public ZicoDataProcessor get(Socket socket, HelloRequest hello) throws IOException { if (hello.getHostname() == null) { log.error("Received HELLO packet with null hostname."); throw new ZicoException(ZicoPacket.ZICO_BAD_REQUEST, "Null hostname."); } HostStore store = getHost(hello.getHostname(), !enableSecurity); if (store == null) { throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized."); } if (store.getAddr() == null || store.getAddr().length() == 0) { store.setAddr(socket.getInetAddress().getHostAddress()); store.save(); } if (enableSecurity) { if (store.getAddr() != null && !store.getAddr().equals(socket.getInetAddress().getHostAddress())) { throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized."); } if (store.getPass() != null && !store.getPass().equals(hello.getAuth())) { throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized."); } } return new ReceiverContext(store); }
函数里面主要做了查找hostStore,并返回和这个hostStore相关的ReceiverContext。ReceiverContext实现了ZicoDataProcessor的process()和commit()方法。
第三部分:分析一下TraceDataService相关
***
HostStore中有search()函数:
public TraceInfoSearchResult search(TraceInfoSearchQuery query) throws IOException { SymbolRegistry symbolRegistry = getSymbolRegistry(); BTreeMapinfos = getInfos(); TraceRecordStore traceDataStore = getTraceDataStore(); TraceRecordStore traceIndexStore = getTraceIndexStore(); if (symbolRegistry == null || infos == null || traceDataStore == null || traceIndexStore == null) { throw new ZicoRuntimeException("Host store " + getName() + " is closed."); } List lst = new ArrayList (query.getLimit()); TraceInfoSearchResult result = new TraceInfoSearchResult(); result.setSeq(query.getSeq()); result.setResults(lst); TraceRecordMatcher matcher = null; int traceId = query.getTraceName() != null ? symbolRegistry.symbolId(query.getTraceName()) : 0; if (query.getSearchExpr() != null) { if (query.hasFlag(TraceInfoSearchQuery.EQL_QUERY)) { matcher = new EqlTraceRecordMatcher(symbolRegistry, Parser.expr(query.getSearchExpr()), 0, 0, getName()); } else if (query.getSearchExpr().length() > 0 && query.getSearchExpr().startsWith("~")) { matcher = new FullTextTraceRecordMatcher(symbolRegistry, TraceRecordSearchQuery.SEARCH_ALL, Pattern.compile(query.getSearchExpr().substring(1))); } else { matcher = new FullTextTraceRecordMatcher(symbolRegistry, TraceRecordSearchQuery.SEARCH_ALL, query.getSearchExpr()); } } // TODO implement query execution time limit int searchFlags = query.getFlags(); boolean asc = 0 == (searchFlags & TraceInfoSearchQuery.ORDER_DESC); Long initialKey = asc ? infos.higherKey(query.getOffset() != 0 ? query.getOffset() : Long.MIN_VALUE) : infos.lowerKey(query.getOffset() != 0 ? query.getOffset() : Long.MAX_VALUE); long tstart = System.nanoTime(); for (Long key = initialKey; key != null; key = asc ? infos.higherKey(key) : infos.lowerKey(key)) { long t = System.nanoTime()-tstart; if ((lst.size() >= query.getLimit()) || (t > MAX_SEARCH_T1 && lst.size() > 0) || (t > MAX_SEARCH_T2)) { result.markFlag(TraceInfoSearchResult.MORE_RESULTS); return result; } TraceInfoRecord tir = infos.get(key); result.setLastOffs(key); if (query.hasFlag(TraceInfoSearchQuery.ERRORS_ONLY) && 0 == (tir.getTflags() & TraceMarker.ERROR_MARK)) { continue; } if (query.getStartDate() != 0 && tir.getClock() < query.getStartDate()) { continue; } if (query.getEndDate() != 0 && tir.getClock() > query.getEndDate()) { continue; } if (traceId != 0 && tir.getTraceId() != traceId) { continue; } if (tir.getDuration() < query.getMinMethodTime()) { continue; } TraceRecord idxtr = (query.hasFlag(TraceInfoSearchQuery.DEEP_SEARCH) && matcher != null) ? traceDataStore.read(tir.getDataChunk()) : traceIndexStore.read(tir.getIndexChunk()); if (idxtr != null) { if (matcher instanceof EqlTraceRecordMatcher) { ((EqlTraceRecordMatcher) matcher).setTotalTime(tir.getDuration()); } if (matcher == null || recursiveMatch(matcher, idxtr)) { lst.add(toTraceInfo(tir, idxtr)); } } } return result; }
//DataReceptionUnitTest中可以看到HostStore中包含一个SymbolRegistry。
//zico在和agent端进行交互时,使用了RESTful风格。