CSE lab4

Part 0 & 1

大多数都是按照文档来，但是碰到小问题卡了接近一天

1. ssh passwordless login 手动建立authorized_keys无效，换用ssh-copy-id能成功改文件但是仍然不能无密码连接
   解决方法：authorized文件必须是600
   sudo vim /etc/ssh/sshd_config 配置文件那一行注释掉
   /etc/init.d/sshd restart 即可
2. 最简情况只需要app的配置文件里面放所有的key就可以，包括当前虚拟机和ta public key

3. TODO try hadoop

Part2

1. 了解一下HDFS架构
   主从架构，Data存内容（若干），Name管记录（只有一个）
2. 需要实现的部分：
   In order to make it compatible with yfs_client, the directory structure part in your namenode must confirm to your design in yfs_client. You should always reuse (call or copy) code in your yfs_client if possible.
   Some block level interface is required to implement a datanode server. HDFS datanode protocol access data block directly with its id, so read_block and write_block should be added to extent_protocol.
   2.1 五个block 层函数

Here is a summary of new extent protocol interface:
- append_block: Given an inode number, allocate and append a block to the inode and return its id. Note that in HDFS, namenode only manage the metadata, so the actual data (even the size) won't be given in this request.
- get_block_ids: Given an inode number, return id of all data blocks of the file.
- read_block: Given a block id, return the data of the disk block.
- write_block: Given a block id and data, write the data to the disk block.
- complete: Given an inode number and size, this request indicates that the writes to the file is finished, so the metedata (file size, modification time) in inode could be updated safely.

1. void inode_manager::append_block(uint32_t inum, blockid_t &bid)
   给一个ino num，分配一个块，给inode然后返回bid
2. 遍历blocks数组
3. 改mtime和ctime

2.2 8个node 层函数

- NameNode::GetBlockLocations: Call get_block_ids and convert block ids to LocatedBlocks.
- NameNode::Complete: Call complete and unlock the file.
- NameNode::AppendBlock: Call append_block and convert block id to LocatedBlock.
- NameNode::Rename: Move a directory entry. Note that src_name/dst_name is entry name, not full path.
- NameNode::Mkdir: Just call mkdir.
- NameNode::Create: Create a file, remember to lock it before return.
- NameNode::Isfile/Isdir/Getfile/Getdir/Readdir/Unlink: The same as the functions in yfs_client, but the framework will call these functions with the locks held, so you shouldn't try to lock them again. Otherwise there will be a deadlock.
- DataNode::ReadBlock/WriteBlock: Call read_block/write_lock to read/write block on extent server. Be careful that client may want to read/write only parts of the block.

1. list NameNode::GetBlockLocations(yfs_client::inum ino)
   extent client 这里的逻辑已经忘得差不多了，用ec->调用get_block_id，然后建立一个list，每个located block是三个变量的struct；遍历list的时候记offset；然后应该要检查一下最后一个block被写了多大
2. bool NameNode::Complete(yfs_client::inum ino, uint32_t new_size)
   调用complete，放锁
3. bool NameNode::Rename(yfs_client::inum src_dir_ino, string src_name, yfs_client::inum dst_dir_ino, string dst_name)
   就是在旧的dir里面找到然后删掉，在新的里面添加一条；找到的话可以用lookup里面的，记得两个list要放回去（好像是unlink还是什么地方有）；注意在list里面删除元素要用erase(iter++)
4. bool NameNode::Mkdir(yfs_client::inum parent, string name, mode_t mode, yfs_client::inum &ino_out)
   看起来只是抄yfs client
5. bool NameNode::Create(yfs_client::inum parent, string name, mode_t mode, yfs_client::inum &ino_out) 什么叫创建文件呢。。看yfs的create
   在返回前要拿自己的锁
6. 六个函数，抄旧的，去掉锁，补充lookup和filename等辅助函数
7. NameNode::LocatedBlock NameNode::AppendBlock(yfs_client::inum ino)
   这个函数太让人费解了，offset是通过getattr拿到的文件大小
8. bool DataNode::ReadBlock(blockid_t bid, uint64_t offset, uint64_t len, string &buf)
   不是文件层面，offset和len都是说的块内