【论文阅读】Twin neural network regression is a semi- supervised regression algorithm

论文下载
GitHub
bib:

@ARTICLE{,
	title 		= {Twin neural network regression is a semi- supervised regression algorithm},
	author 		= {Sebastian J Wetzel and Roger G Melko and Isaac Tamblyn},
	journal 	= {Machine Learning: Science and Technology},
	year 		= {2022},
	volume 		= {3},
	number 		= {4},
	pages 	    = {045007},
	doi         = {10.1088/2632-2153/ac9885}
}

1. 摘要

Twin neural network regression (TNNR) is trained to predict differences between the target values of two different data points rather than the targets themselves.

TNNR是作者以前的工作，但是针对回归问题的，这篇论文针对的是半监督回归（semi-supervised regression）。

By ensembling predicted differences between the targets of an unseen data point and all training data points, it is possible to obtain a very accurate prediction for the original regression problem.

通过集成未见过的数据点的目标与所有训练数据点之间的预测差异，可以获得对原始回归问题的非常准确的预测。这一步应该是描述对于不同差异的平均操作。

Since any loop of predicted differences should sum to zero, loops can be supplied to the training data, even if the data points themselves within loops are unlabelled.

loop of predicted differences should sum to zero 这一步在上一篇论文中有介绍。

Semi-supervised training improves TNNR performance, which is already state of the art, significantly.

Note:
前置工作《Twin Neural Network Regression》。这篇工作是原作者方法对于半监督学习方向的拓展。

2. 算法描述

$$\begin{array}{rl}{y}_j^{pre}& =\frac{1}{m}\sum _{i=1}^m((F(x_j,x_i^{train})+y_i^{traian})\\ & =\frac{1}{m}\sum _{i=1}^m((\frac{1}{2}F(x_j,x_i^{train})+\frac{1}{2}F(x_i^{train},x_j)+y_i^{traian})\end{array}\text{\hspace{1em}(1)}$$, where

• $x_i^{train}\in X^{train}$，表示训练集中的训练样本。

$n$ labeled data and $m$ unlabeled data
有标记数据损失:
$${\ell }_{MSE}=\frac{1}{n^2}\sum _{ij}(F(x_i,x_j)-(y_i-y_j){)}^2$$

注意$n^2$，表示任意两个有标记数据配对。这也是天然适合半监督场景，拓展了训练数据集。

无标记数据损失：
$${\ell }_{loop}=\frac{1}{(n+m{)}^2}\sum _{ijk}(F(x_i,x_j)+F(x_j,x_k)+F(x_k,x_i))$$

总损失：
$$\ell ={\ell }_{MSE}+\Lambda {\ell }_{loop}$$

上面已经把故事讲清楚了，但是还有一些细节。

• 在mini-batch中，数据是如何配对的。
• $\Lambda$ 对于实验效果的影响。
• loop type对于结果的影响。

3. 实验

论文中的对比实验中，对比算法较少。