投稿任务告与段落了,最终的结果是被TMC给early reject了。这神一样的审稿意见让我真的是老头地铁看手机啊!所以虽然TMC没有给我rebuttal的机会。所以我还是打算在CSDN进行一次rebuttal。
其实我做的东西很简单,就是把时间序列中的将时间序列转化为图的思想应用到无线定位领域。以下是TMC的审稿意见:
Editor Comments
Associate Editor
Comments to the Author:
For your paper I got three independent reviews from experts in various fields. All the Reviewers made a plethora of major and minor concerns. After my own reading, my concerns are as follows:
Reviewer Comments
Please note that some reviewers may have included additional comments in a separate file. If a review contains the note “see the attached file” under Section III A – Public Comments, you will need to log on to ScholarOne Manuscripts to view the file. After logging in, select the Author Center, click on the “Manuscripts with Decisions” queue and then clicking on the “view decision letter” link for this manuscript. You must scroll down to the very bottom of the letter to see the file(s), if any. This will open the file that the reviewer(s) or the associate editor included for you along with their review.
Reviewer: 1
Recommendation: Author Should Prepare A Major Revision For A Second Review
Comments:
Additional Questions:
Which category describes this manuscript?: Research/Technology
How relevant is this manuscript to the readers of this periodical? Please explain under Public Comments below.: Relevant
Please explain how this manuscript advances this field of research and/or contributes something new to the literature.: The manuscript addresses the limitation of existing wireless sensing methods by utilizing signal correlation in channel, time, and space. The experimental results indicate the effectiveness of the proposed framework in improving the accuracy and robustness of wireless sensing applications, and therefore contributes a new approach to the literature.
Is the manuscript technically sound? Please explain under Public Comments below.: Appears to be - but didn’t check completely
Are the title, abstract, and keywords appropriate? Please explain under Public Comments below.: Yes
Does the manuscript contain sufficient and appropriate references? Please explain under Public Comments below.: Important references are missing; more references are needed
If you are suggesting additional references they must be entered in the text box provided. All suggestions must include full bibliographic information plus a DOI.
If you are not suggesting any references, please type NA.: 1) Xifilidis, Theofanis, and Kostas E. Psannis. “Correlation-based wireless sensor networks performance: The compressed sensing paradigm.” Cluster Computing (2022): 1-17.
Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? Please explain under Public Comments below.: Yes
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic? Please explain under Public Comments below.: Could be improved
Please rate the readability of this manuscript. Please explain your rating under Public Comments below.: Easy to read
Should the supplemental material be included? (Click on the Supplementary Files icon to view files): Yes, as part of the digital library for this submission if accepted
If yes to 6, should it be accepted: As is
If this manuscript is an extended version of a conference publication, does it offer substantive novel contributions beyond those of the previously published work(s)- i.e. expansion of key ideas, examples, elaborations etc. New results are not required: Not applicable
Please rate the manuscript. Please explain under Public Comments below.: Fair
Reviewer: 2
Recommendation: Reject
Comments:
In this manuscript, the authors study an interesting problem, i.e., learning correlation among wireless signals via graph neural network. They propose a Signal Correlation Learning (SCL) framework which represents three types of correlation among wireless sensors and apply this framework to different learning tasks as well as different wireless sensors. However, the paper does have the following issues.
First, the contribution of the paper is not that significant. Some main ideas such as channel attention, graph construction, or graph attention have already appeared in existing works. Although the authors made some change to them, the explanation of the change is not clear and the evaluation is insufficient. For example:
Second, some settings of the experiments are questionable. For anomaly detection, it is unclear what kind of anomalies are added into the data. For localization, it is unclear how big the test site is. From Fig. 8, it seems that the test site is quite small, far different from the real-world indoor localization scenario. Without knowing such information, it is difficult to judge whether the proposed framework can work in practice.(异常检测场景作为一个case study,是说除了定位我们还能做异常检测。至于详细的说明,兄弟RFID克隆攻击?这个说明还不够详细的吗?你可是TMC的审稿人啊!!RFID中的伪造标签不知道?你就算不知道我后面不是给了参考文献?你去看一下啊!你要是觉得这东西我应该在论文里面写出来,那我建议你也去和第一个审稿人打一架,他说我写太多了,14页了。至于场景小,那TM NFC的全称是什么,不就是进场通信吗?RFID的通信距离有多远?)
Moreover, some related works about the application of graph neural network in localization or anomaly detection are not mentioned in the paper, such as:
Guan, Siwei, Binjie Zhao, Zhekang Dong, Mingyu Gao, and Zhiwei He. “GTAD: Graph and Temporal Neural Network for Multivariate Time Series Anomaly Detection.” Entropy 24, no. 6 (2022): 759.(这篇论文我要是引了我把你眼珠子挖出来可以不,打个赌不?)
Zheng, Han, Yan Zhang, Lan Zhang, Hao Xia, Shaojie Bai, Guobin Shen, Tian He, and Xiangyang Li. “Grafin: An applicable graph-based fingerprinting approach for robust indoor localization.” In 2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS), pp. 747-754. IEEE, 2021.
Luo, Xuanshu, and Nirvana Meratnia. “A Geometric Deep Learning Framework for Accurate Indoor Localization.” In 2022 IEEE 12th International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1-8. IEEE, 2022.
(以上两篇论文没有引述其实是一个很麻烦的事情,因为我这个论文最早完工的时候是前年,当时我发现GNN可以应用于无线感知的各个领域而且应该可以取得一个不错的效果,其实这是一个很简单的思想,无线感知经常使用参考信号,会根据已知信号和未知信号之间的关系进行推理,那很自然的想法是我们能不能用一些方法去自动挖掘出信号之间的关系呢?那很容易想到GNN,之后做出来的时候投了NSDI被拒了,后面会放上NSDI的审稿意见。然后拖了大半年改投TMC直到现在才出审稿意见。所以这么长的时间跨度肯定也有人做出类似的工作,所以没有办法引用哪些论文了。)
Also, some typical deep graph neural networks such as the graph attention networks [14] are not included in the baseline.
(哈哈哈哈哈,我的baseline里面有两篇时间序列为图的经典论文一个是图差分网络GDN、和MTAD。这两个都使用了GAT作为骨干网络,怎么能叫没有使用GAT作为baseline呢?并且我在baseline介绍里面已经说的很清楚了。你是看不懂还是看不到吗?)
Finally, there are quite a lot typos in the paper, which may confuse the readers. For instance:
Additional Questions:
Which category describes this manuscript?: Research/Technology
How relevant is this manuscript to the readers of this periodical? Please explain under Public Comments below.: Relevant
Please explain how this manuscript advances this field of research and/or contributes something new to the literature.: This manuscript introduces a signal correlation learning framework that represents the signal and learns the relationship of wireless sensors via a graph neural network. This framework can be applied to different sensing problems including localization, anomaly detection, and human activity recognition with different wireless sensors including RFID and Bluetooth.
Is the manuscript technically sound? Please explain under Public Comments below.: Appears to be - but didn’t check completely
Are the title, abstract, and keywords appropriate? Please explain under Public Comments below.: Yes
Does the manuscript contain sufficient and appropriate references? Please explain under Public Comments below.: Important references are missing; more references are needed
If you are suggesting additional references they must be entered in the text box provided. All suggestions must include full bibliographic information plus a DOI.
If you are not suggesting any references, please type NA.: Guan, Siwei, Binjie Zhao, Zhekang Dong, Mingyu Gao, and Zhiwei He. “GTAD: Graph and Temporal Neural Network for Multivariate Time Series Anomaly Detection.” Entropy 24, no. 6 (2022): 759.
Zheng, Han, Yan Zhang, Lan Zhang, Hao Xia, Shaojie Bai, Guobin Shen, Tian He, and Xiangyang Li. “Grafin: An applicable graph-based fingerprinting approach for robust indoor localization.” In 2021 IEEE 27th International Conference on Parallel and Distributed Systems (ICPADS), pp. 747-754. IEEE, 2021.
Luo, Xuanshu, and Nirvana Meratnia. “A Geometric Deep Learning Framework for Accurate Indoor Localization.” In 2022 IEEE 12th International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 1-8. IEEE, 2022.
Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? Please explain under Public Comments below.: Yes
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic? Please explain under Public Comments below.: Could be improved
Please rate the readability of this manuscript. Please explain your rating under Public Comments below.: Readable - but requires some effort to understand
Should the supplemental material be included? (Click on the Supplementary Files icon to view files): Does not apply, no supplementary files included
If yes to 6, should it be accepted:
If this manuscript is an extended version of a conference publication, does it offer substantive novel contributions beyond those of the previously published work(s)- i.e. expansion of key ideas, examples, elaborations etc. New results are not required: Not applicable
Please rate the manuscript. Please explain under Public Comments below.: Fair
Reviewer: 3
Recommendation: Reject
Comments:
The paper presents and discusses a Signal Correlation Learning (SCL) framework adopted to define a graph that represents the signal correlation of multiple wireless sensors. My comments after reading the paper are as follows:
Additional Questions:
Which category describes this manuscript?: Research/Technology
How relevant is this manuscript to the readers of this periodical? Please explain under Public Comments below.: Relevant
Please explain how this manuscript advances this field of research and/or contributes something new to the literature.: There are questions about the novelty of the approach.
Is the manuscript technically sound? Please explain under Public Comments below.: Appears to be - but didn’t check completely
Are the title, abstract, and keywords appropriate? Please explain under Public Comments below.: Yes
Does the manuscript contain sufficient and appropriate references? Please explain under Public Comments below.: References are sufficient and appropriate
If you are suggesting additional references they must be entered in the text box provided. All suggestions must include full bibliographic information plus a DOI.
If you are not suggesting any references, please type NA.: NA
Does the introduction state the objectives of the manuscript in terms that encourage the reader to read on? Please explain under Public Comments below.: Could be improved
How would you rate the organization of the manuscript? Is it focused? Is the length appropriate for the topic? Please explain under Public Comments below.: Could be improved
Please rate the readability of this manuscript. Please explain your rating under Public Comments below.: Easy to read
Should the supplemental material be included? (Click on the Supplementary Files icon to view files): Does not apply, no supplementary files included
If yes to 6, should it be accepted:
If this manuscript is an extended version of a conference publication, does it offer substantive novel contributions beyond those of the previously published work(s)- i.e. expansion of key ideas, examples, elaborations etc. New results are not required: Not applicable
Please rate the manuscript. Please explain under Public Comments below.: Fair
综上这次TMC投稿给我的感觉非常的差,先是审稿申了这么长时间,其次就这审稿意见甚至比不上chatpaper自动生成的。一个图神经网络的论文,三个意见包括副主编的意见居然和图神经网络没有任何关系。真的审成这样也是没谁了。真的我看网上喷好多人说 infocom、AAAI、IJCAI水,就是因为审稿人水平差,利益相关者。但是我这次一看TMC这种臭水平的垃圾审稿人,真的别喷infocom AAAI了,我现在才知道原来期刊的水比会议深太多了。对了我之前投过NSDI,以下是NSDI的审稿意见:
Overall merit
1.
Reject (I will argue to reject this paper)
Reviewer expertise
1.
No familiarity
Paper summary
The paper proposes to use signals–collected by one device about multiple other devices–to construct a signal correlation learning framework which can then be used to solve multiple sensing tasks.
Comments for authors
The tasks you describe, e.g., anomaly detection, localization, and human activity recognition, are very challenging and require novel solutions. The idea of taking advantage of the correlated signals is also excellent. However, this reviewer does not think this is the first time someone is proposing to take advantage of correlated signals. For example, Emerald – a company that is using wireless signals to infer health signals – is using correlated wireless data to do so. Moreover, the correlation of signals is used to improve wireless data communication etc.
As such this reviewer is confused by some of the claims of the authors. Overall, this paper seems to be much more focused on explaining the ML setup of the system rather than the application side. As such it is unclear why NSDI is the best conference for such a paper.
Another aspect that the paper does not deal with is that wireless signals are constantly changing, e.g., because a device is moved, because of temperature, because of a human moving, because of another device that is active (e.g., the microwave) etc. In general, the data sets that you are using are rather small and do not span long time periods. You claim that your system is self-adaptive but you never challenge it.
Indeed, before you can even discuss your SCL framework you should first state what kind of data you are looking at. Next, you should explain your assumptions about the problems you want to solve as well as the data. Next, you should explain what the components of your model are, e.g., what is the meaning of a node and a graph, and how does time play into this. For example, up to page 4, you talk about a graph but never explained what your nodes are…
You also talk about a signal correlation graph but you do not talk about how you measure correlation. In particular, you do not talk about how you explicitly caption the three different forms of correlation. Also, what is the complexity of handling the correlation?
Regarding your results, you claim a factor of up to 2.5 improvements over the baseline… But the relevant number is the improvement over the best alternative. This seems to be significantly smaller.
Review #503B
Overall merit
1.
Reject (I will argue to reject this paper)
Reviewer expertise
4.
Expert
Paper summary
The paper proposes a Signal Correlation Learning (SCL) framework, a data-driven method based on Graph Neural Network (GNN) to find signal correlations in various sensing tasks. SCL constructs a direct graph to represent the channel-, spatial-, and temporal- signal correlations of wireless sensors. The framework preprocessed data, assigns weight, constructs a graph using KL-based method, then correlates features. The method is evaluated on RFID/bluetooth localization, anomaly detection, and human sensing to demonstrate its performance.
Comments for authors
It’s interesting to see work that tries to build general approaches for wireless sensing. However, it was difficult for me to understand the practical problem that this paper is trying to solve. Is the goal to come up with more elegant solutions to existing problems? Or is the goal to outperform prior systems? It seems that the paper is trying to do the former but is pitching the latter, yet I believe it underperforms in comparison to prior art.
The paper focuses on 3 tasks, and says it outperforms prior art by 1.5-2.5x. I’m not sure that is true for any of the tasks:
If you take RFID localization, the paper achieves 20cm accuracy (Fig. 9b), but state-of-the-art systems achieve 1-cm of accuracy. So, the paper underperforms by 20x. In comparison to an old baseline like Landmarc from the early 2000’s, it achieves better performance, but that’s not an acceptable baseline
If you take activity recognition, the paper only studies 3 activities. State-of-the-art systems can either precisely recover trajectories/movements (e.g., work from Dina Katabi’s group) or can classify at least a dozen movements. So, I’m not sure the paper outperforms there either.
Finally, the anomaly detection task was unclear. The paper states that they “invade the data and replace them with different signals to simulate the anomaly attacking” - what does this mean? Is this a problem to solve?
Overall, exploring more elegant or simpler solutions to exiting problems/tasks might be intellectually interesting, but it’s not clear to me that this paper does that either, and it seems to underperform in comparison to prior art.
Few other detailed comments:
The paper states that SCL needs multiple sensors to coexist to work. I wonder how many sensors do we minimally need for the SCL to work? The paper doesn’t mention this and only states “several” E41C Impinj tags, instead of an actual number.
What’s the computation complexity for the method? SCL is currently using a high-performance GPU server to run the inference. I could imagine that many applications would like to run the wireless sensing application on edge devices. Therefore, a brief understanding of the computing complexity, or the latency of the whole inference task will give us a better understanding of the framework.
From the evaluation of SCL on RFID and Bluetooth systems on different sensing tasks, I didn’t see much difference between them. I wonder why we need three different channels (RSSI, Phase, Doppler) for RFID while Bluetooth only needs RSSI and achieve almost the same accuracy.
Some typos:
There are some typos in the paper that wrote SLC instead of SCL.
On page two, in this sentence: “In this paper, we perform the weighted average operation on the the feature”, it uses two “the”.
Review #503C
Overall merit
2.
Weak reject (I think it should be rejected, but I am fine if others want to accept.)
Reviewer expertise
2.
Some familiarity
Paper summary
This paper presents a correlation learning mechanism to improve sensing accuracy in IoT deployments. The authors evaluate their framework in a testbed and find 1.5-2.5x improvement in sensing accuracy relative to systems employing existing techniques.
Comments for authors
Thank you for submitting to NSDI.
While the claimed improvements and novelty sound significant, I have a primary concern over the presentation:
for a systems conference, there is insufficient overview of the techniques employed and their novelty. The paper unfortunately goes straight into the math without any description of the protocol in operation.
related, beyond the insight of using signal correlation, what is hard about the work? Is this a straightforward adoption of signal correlation? Or is their insight into the approach as well.
while there is some discussion of related work, I would have liked to have seen more on the employment of signal correlation from related areas.
this is a problem beyond this paper, but the evaluation is fairly ad hoc. While detailed, the challenge with IoT is that the deployments are rather heterogeneous/varied. Does this work solve a practical problem in the field? Or is it speculating about challenges that might come along in the future?
看到没有,不对比就没有差距。而且咱也不是那种接受不了别人的批评那种,NSDI虽然给我拒了但是我心服口服啊!人家审稿人看不懂的地方就直说我看不懂。然后人家就比较人家能看懂的地方是否符合会议的要求。最终认为不符合给拒绝了。这样我也认了。这才叫专业的审稿人啊!谦虚、客观、有礼貌、就事论事、没有攻击性。这样一比那帮TMC的臭鱼烂虾怎么有脸活下去呢?