zico源代码分析(一) 数据接收和存储部分

zorka和zico的代码地址:https://github.com/jitlogic

由于zico是zorka的collecter端,所以在介绍zico之前首先说一下zorka和数据结构化存储和传输的部分。zorka抓取到数据后,会封装成一条TraceRecord,TraceRecord中包含本条trace的类,方法,调用链等信息,但是这些信息都是将对应的ID存在TraceRecord,而ID对应的详细信息会存在Symbol中,也就是说Symbol相当于一个数据字典。在传送和存储的之前会进行打TAG操作,已经来标记一条信息是Symbol的一个条目或者是TraceRecord中的一个条目。

 

首先,先试着理一下Zico工作的主要步骤:

ZicoServer通过socket建立连接,在run()方法中新建了ZicoServerConnector(继承自ZicoConnector),ZicoServerConnector中的核心部分是runCycle()函数。

复制代码
//此方法在run方法中被调用
    private void runCycle() throws IOException {
        ZicoPacket pkt = recv();
        switch (pkt.getStatus()) {
            case ZICO_PING: {
                send(ZICO_PONG);
                break;
            }
            case ZICO_HELLO: {
                List lst = ZicoCommonUtil.unpack(pkt.getData());
                log.debug("Encountered ZICO HELLO packet: " + lst + "(addr=" + saddr + ")");
                if (lst.size() > 0 && lst.get(0) instanceof HelloRequest) {
                    context = factory.get(socket, (HelloRequest) lst.get(0));
                    send(ZICO_OK);
                } else {
                    log.error("ZICO_HELLO packet with invalid content: " + lst + "(addr=" + addr + ")");
                    send(ZICO_BAD_REQUEST);
                }
                break;
            }
            case ZICO_DATA: {
                //log.debug("Received ZICO data packet from " + saddr + ": status=" + pkt.getStatus()
                //        + ", dlen=" + pkt.getData().length);
                if (context != null) {
                    for (Object o : ZicoCommonUtil.unpack(pkt.getData())) {
                        context.process(o);
                    }
                    context.commit();
                    send(ZICO_OK);
                } else {
                    log.error("Client " + saddr + " not authorized.");
                    send(ZICO_AUTH_ERROR);
                }
                break;
            }
            default:
                log.error("ZICO packet from " + saddr + " with invalid status code: " + pkt.getStatus());
                send(ZICO_BAD_REQUEST);
                break;
        }
    } 
   
复制代码

recv()函数是将通过socket读取的数据封装成一个ZicoPacket对象返回。在连接建立的初期,zico和zorka之间会发送ping-pong报文以及hello报文来测试建立的连接。ZicoPacket中的getData()方法返回一个byte数组,ZicoCommonUtil中的unpack()方法会将数据从byte数组中解包(unpack list of fressian-encoded objects from byte array)。unpack()函数执行的返回结果是List,ZicoDataProcessor将对这个List执行process()方法和commit()方法。

 

只有ReceiverContext实现了ZicoDataProcessor接口,下面我们看一下ReceiverContext这个类。

复制代码
 @Override
    public synchronized void process(Object obj) throws IOException {

        if (hostStore.hasFlag(HostInfo.DELETED)) {
            log.info("Resetting connection for " + hostStore.getName() + " due to dirty SID map.");
            throw new ZicoException(ZicoPacket.ZICO_EOD,
                    "Host has been deleted. Connection needs to be reset. Try again.");
        }

        if (hostStore.hasFlag(HostInfo.DISABLED)) {
            // Host store is disabled. Ignore incoming packets.
            return;
        }

        if (dirtySidMap) {
            log.info("Resetting connection for " + hostStore.getName() + " due to dirty SID map.");
            throw new ZicoException(ZicoPacket.ZICO_EOD,
                "Host was disabled, then enabled and SID map is dirty. Resetting connection.");
        }

        try {
            if (obj instanceof Symbol) {
                processSymbol((Symbol) obj);
            } else if (obj instanceof TraceRecord) {
                processTraceRecord((TraceRecord) obj);
            } else {
                if (obj != null) {
                    log.warn("Unsupported object type:" + obj.getClass());
                } else {
                    log.warn("Attempted processing NULL object (?)");
                }
            }
        } catch (Exception e) {
            log.error("Error processing trace record: ", e);
        }
    }
复制代码

ReceiverContext中包含一个HostStore(Represents performance data store for a single agent)对象。

下面,我们看一下ReceiverContext中的processTraceRecord()方法。

复制代码
private void processTraceRecord(TraceRecord rec) throws IOException {
        if (!hostStore.hasFlag(HostInfo.DISABLED)) {
            rec.traverse(this);
            visitedObjects.clear();
            hostStore.processTraceRecord(rec);
        } else {
            log.debug("Dropping trace for inactive host: " + hostStore.getName());
        }
    } 
复制代码

此方法中调用了TraceRecord的tracerse方法。

复制代码
   public void traverse(MetadataChecker checker) throws IOException {
        classId = checker.checkSymbol(classId, this);
        methodId = checker.checkSymbol(methodId, this);
        signatureId = checker.checkSymbol(signatureId, this);

        if (exception instanceof SymbolicException) {
            ((SymbolicException) exception).traverse(checker);
        }

        if (attrs != null) {
            Map newAttrs = new LinkedHashMap();
            for (Map.Entry e : attrs.entrySet()) {
                newAttrs.put(checker.checkSymbol(e.getKey(), this), e.getValue());
            }
            attrs = newAttrs;
        }

        if (children != null) {
            for (TraceRecord child : children) {
                child.traverse(checker);
            }
        }

        if (marker != null && 0 != (flags & TRACE_BEGIN)) {
            marker.traverse(checker);
        }
    }
复制代码

获取了三个id是****,读取了traceRecord及其孩子的attrs(Attributes grabbed at this method execution (by spy instrumentation engine))。

  在processTraceRecord()方法中调用了HostStore类中的ProcessTraceRecord(TraceRecord obj)方法,在其中将TraceRecord(Represents trace information about single method execution,May contain references to information about calls from this method)包装成了TraceInfoRecord,TraceInfoRecord比TraceRecord多了四个属性:dataOffs,dataLen,indexOffs,indexLen。个人猜测用来控制块操作的。

  在HostStore类的processTraceRecord()方法中将TraceRecord进行了更加详细的解析,将TraceRecord的对象分解为DataSore,IndexStore,infos等并且分别写入文件。

复制代码
    public void processTraceRecord(TraceRecord rec) throws IOException {

        TraceRecordStore traceDataStore = getTraceDataStore();
        TraceRecordStore traceIndexStore = getTraceIndexStore();
        BTreeMap infos = getInfos();
        Map tids = getTids();

        if (traceDataStore == null || traceIndexStore == null || infos == null || tids == null
                || hasFlag(HostInfo.DISABLED|HostInfo.DELETED)) {
            throw new ZicoRuntimeException("Store " + getName() + " is closed and cannot accept records.");
        }

        TraceRecordStore.ChunkInfo dchunk = traceDataStore.write(rec);

        List tmp = rec.getChildren();

        int numRecords = ZicoUtil.numRecords(rec);

        rec.setChildren(null);
        TraceRecordStore.ChunkInfo ichunk = traceIndexStore.write(rec);
        rec.setChildren(tmp);


        TraceInfoRecord tir = new TraceInfoRecord(rec,numRecords,
                dchunk.getOffset(), dchunk.getLength(),
                ichunk.getOffset(), ichunk.getLength(),
                rec.getAttrs() != null ? ZicoUtil.toIntArray(rec.getAttrs().keySet()) : null);

        checkAttrs(tir);

        infos.put(tir.getDataOffs(), tir);

        int traceId = tir.getTraceId();

        if (!tids.containsKey(traceId)) {
            tids.put(traceId, symbolRegistry.symbolName(traceId));
        }

    }
复制代码

  到此,总结一下数据从sokect读取到写入文件的过程,ZicoServerConnector中负责根据socket生成inputStream从socket中读取数据,此时读取的数据是fressian编码的byte数组,ZicoCommonUtil类的unpack()方法将这个byte数组解析为一个List,ReceiverContext类进一步将这些Object区分为Symbol和TraceRecord,而HostStore进一步将TraceRecord解析为更加详细的信息,也就是对应下面的目录中的信息。

zico将收到的agent端的数据存储在zico/data文件下,目录结构如下图所示:

tdat中***存储的是host列表。

tidx:***(还不清楚)

host.properties中存储着host相关的信息(name,IP,group等等)

symbol.dat***

trace.db****

 

HostStore类代表一个agent的性能数据的存储(Represents performance data store for a single agent)。在HostStore类的构造函数中调用了open()函数,我们可以根据以下代码分析目录结构

复制代码
    public synchronized void open() {
        try {
            load();
            if (symbolRegistry == null) {
                symbolRegistry = new PersistentSymbolRegistry(
                    new File(ZicoUtil.ensureDir(rootPath), "symbols.dat"));
            }

            if (traceDataStore == null) {
                traceDataStore = new TraceRecordStore(config, this, "tdat", 1, this);
            }

            if (traceIndexStore == null) {
                traceIndexStore = new TraceRecordStore(config, this, "tidx", 4);
            }

            db = dbf.openDB(ZorkaUtil.path(rootPath, "traces.db"));

             //private BTreeMap infos;
             infos = db.getTreeMap(DB_INFO_MAP);
             //private Map tids;
             tids = db.getTreeMap(DB_TIDS_MAP);
             //private BTreeMap> attrs;
             attrs = db.getTreeMap(DB_ATTR_MAP);

        } catch (IOException e) {
            log.error("Cannot open host store " + name, e);
        }
    }
复制代码

      PersistentSymbolRegistry类继承自SymbolRegistry类,而SymbolRegistry类中包含了两个Map(symbolIds以及symbolNames)。PersistentSymbolRegistry类中的open()方法会从symbol.dat文件中读取symbol,并put进symbolIds和symbolNames。

      TraceReocrdStore类在其构造函数中会新建一个RDSSotre(Raw Data Store (RDS) builds upon RAGZ stream classes)对象。在RSSsotre的open()方法中,会建立RAGZInputStream和RAGZOutputStream来读写RAGZSegment(represents a single in-memory segment)。RAGZSegment类有unpack()方法对数据进行解压,此处调用了java.util.zip包。

  在zico-core包的pom文件中, 有以下语句

        
            org.mapdb
            mapdb
            ${mapdb.version}
        
       

  此处将mapdb导入了maven项目。对于trace.db文件使用了org.parboiled.BaseParser。  http://parboiled.org。这是一个轻量级no-SQL数据库

 

 第二部分

zico使用了google guice IOC,在ProdZicoModule中有如下语句

复制代码
@Override
    public void configure(Binder binder) {
        super.configure(binder);
        binder.bind(UserManager.class).asEagerSingleton();
        binder.bind(UserContext.class).to(UserHttpContext.class);
        binder.bind(DBFactory.class).to(FileDBFactory.class);
        binder.bind(ZicoDataProcessorFactory.class).to(HostStoreManager.class);
    }
复制代码

先分析最后一句话,ZicoDataProcessorFactory接口中只有一个get方法,此方法在HostStoreManager方法中实现如下

复制代码
  @Override
    public ZicoDataProcessor get(Socket socket, HelloRequest hello) throws IOException {

        if (hello.getHostname() == null) {
            log.error("Received HELLO packet with null hostname.");
            throw new ZicoException(ZicoPacket.ZICO_BAD_REQUEST, "Null hostname.");
        }

        HostStore store = getHost(hello.getHostname(), !enableSecurity);

        if (store == null) {
            throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized.");
        }

        if (store.getAddr() == null || store.getAddr().length() == 0) {
            store.setAddr(socket.getInetAddress().getHostAddress());
            store.save();
        }

        if (enableSecurity) {
            if (store.getAddr() != null && !store.getAddr().equals(socket.getInetAddress().getHostAddress())) {
                throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized.");
            }

            if (store.getPass() != null && !store.getPass().equals(hello.getAuth())) {
                throw new ZicoException(ZicoPacket.ZICO_AUTH_ERROR, "Unauthorized.");
            }
        }

        return new ReceiverContext(store);
    }
复制代码

函数里面主要做了查找hostStore,并返回和这个hostStore相关的ReceiverContext。ReceiverContext实现了ZicoDataProcessor的process()和commit()方法。

 

 第三部分:分析一下TraceDataService相关

***

HostStore中有search()函数:

复制代码
 public TraceInfoSearchResult search(TraceInfoSearchQuery query) throws IOException {

        SymbolRegistry symbolRegistry = getSymbolRegistry();
        BTreeMap infos = getInfos();
        TraceRecordStore traceDataStore = getTraceDataStore();
        TraceRecordStore traceIndexStore = getTraceIndexStore();

        if (symbolRegistry == null || infos == null || traceDataStore == null || traceIndexStore == null) {
            throw new ZicoRuntimeException("Host store " + getName() + " is closed.");
        }

        List lst = new ArrayList(query.getLimit());

        TraceInfoSearchResult result = new TraceInfoSearchResult();
        result.setSeq(query.getSeq());
        result.setResults(lst);

        TraceRecordMatcher matcher = null;

        int traceId = query.getTraceName() != null ? symbolRegistry.symbolId(query.getTraceName()) : 0;

        if (query.getSearchExpr() != null) {
            if (query.hasFlag(TraceInfoSearchQuery.EQL_QUERY)) {
                matcher = new EqlTraceRecordMatcher(symbolRegistry,
                        Parser.expr(query.getSearchExpr()),
                        0, 0, getName());
            } else if (query.getSearchExpr().length() > 0 && query.getSearchExpr().startsWith("~")) {
                matcher = new FullTextTraceRecordMatcher(symbolRegistry,
                        TraceRecordSearchQuery.SEARCH_ALL, Pattern.compile(query.getSearchExpr().substring(1)));
            } else {
                matcher = new FullTextTraceRecordMatcher(symbolRegistry,
                        TraceRecordSearchQuery.SEARCH_ALL, query.getSearchExpr());
            }
        }

        // TODO implement query execution time limit

        int searchFlags = query.getFlags();

        boolean asc = 0 == (searchFlags & TraceInfoSearchQuery.ORDER_DESC);

        Long initialKey = asc
                ? infos.higherKey(query.getOffset() != 0 ? query.getOffset() : Long.MIN_VALUE)
                : infos.lowerKey(query.getOffset() != 0 ? query.getOffset() : Long.MAX_VALUE);

        long tstart = System.nanoTime();

        for (Long key = initialKey; key != null; key = asc ? infos.higherKey(key) : infos.lowerKey(key)) {

            long t = System.nanoTime()-tstart;

            if ((lst.size() >= query.getLimit()) || (t > MAX_SEARCH_T1 && lst.size() > 0) || (t > MAX_SEARCH_T2)) {
                result.markFlag(TraceInfoSearchResult.MORE_RESULTS);
                return result;
            }

            TraceInfoRecord tir = infos.get(key);

            result.setLastOffs(key);

            if (query.hasFlag(TraceInfoSearchQuery.ERRORS_ONLY) && 0 == (tir.getTflags() & TraceMarker.ERROR_MARK)) {
                continue;
            }

            if (query.getStartDate() != 0 && tir.getClock() < query.getStartDate()) {
                continue;
            }

            if (query.getEndDate() != 0 && tir.getClock() > query.getEndDate()) {
                continue;
            }

            if (traceId != 0 && tir.getTraceId() != traceId) {
                continue;
            }

            if (tir.getDuration() < query.getMinMethodTime()) {
                continue;
            }

            TraceRecord idxtr = (query.hasFlag(TraceInfoSearchQuery.DEEP_SEARCH) && matcher != null)
                    ? traceDataStore.read(tir.getDataChunk())
                    : traceIndexStore.read(tir.getIndexChunk());

            if (idxtr != null) {
                if (matcher instanceof EqlTraceRecordMatcher) {
                    ((EqlTraceRecordMatcher) matcher).setTotalTime(tir.getDuration());
                }
                if (matcher == null || recursiveMatch(matcher, idxtr)) {
                    lst.add(toTraceInfo(tir, idxtr));
                }
            }

        }

        return result;
    }
复制代码

 

 

 

//DataReceptionUnitTest中可以看到HostStore中包含一个SymbolRegistry。

 //zico在和agent端进行交互时,使用了RESTful风格。

你可能感兴趣的:(Java,APM)