Github每日精选(第41期):跨端数据同步加密工具restic

restic

restic 是一个快速、安全、高效的备份工具,有的时候我们在做服务端的时候,有些文件我们需要进行另外的保存,可能是因为这是重要的文件需要进行安全的备份,也可能是因为文件的私密性。

在安全方面上restic 做得不错,但是他就失去了像网盘那样的便利性,这也是restic 为啥只在一部分人里面流行而已。

Github每日精选(第41期):跨端数据同步加密工具restic_第1张图片

github的地址在这里。

restic 有几个优势:

  • 密码登陆和数据加密
  • 版本管理,像git一样,他的仓库可以还原到最初的版本,我们需要哪个版本都可以通过像git那样还原回来这就最大的方便了。
  • 开源免费,跨平台可用。

安装

在 Debian 上,可以从官方仓库安装的软件包restic ,apt-get:

$ apt-get install restic

使用

在下面的使用过程中,如果你使用过git的代码管理的话,你会发现,resticgit很像,甚至是一样的,一样的命令。

创建仓库

初始化一个仓库,按提示输入密码,这个密码打死也不能告诉别人,但是你也不能自己忘掉,忘掉的话这个仓库基本是毁了。

restic init --repo /tmp/backup

提示我们输入两次代码:

enter password for new repository: 
enter password again: 
created restic repository 25d29858c9 at /tmp/backup

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.

告诉我们,这个密码很重要,不要丢了。

备份文件

这样我们就有一个仓库/tmp/backup了,有了仓库以后,我们就可以把想备份的文件放到这个仓库来了。

restic --repo /tmp/backup backup manim/

这里我们要备份当前文档下的manim 文件夹。提示我们输入密码:

enter password for repository: 
repository 25d29858 opened successfully, password is correct
created new cache in /root/.cache/restic
no parent snapshot found, will read all files

Files:         207 new,     0 changed,     0 unmodified
Dirs:           57 new,     0 changed,     0 unmodified
Added to the repo: 75.624 MiB

processed 207 files, 75.530 MiB in 0:01
snapshot 5cc831ec saved

简单的列出了我们这个文件夹里的相关信息,还有我们备份的版本号为5cc831ec

我们在备份另外一个压缩文件夹。

restic --repo /tmp/backup backup gotty_linux_amd64.tar.gz

显示的信息如下:

enter password for repository: 
repository 25d29858 opened successfully, password is correct
no parent snapshot found, will read all files

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Added to the repo: 2.815 MiB

processed 1 files, 2.814 MiB in 0:00
snapshot 37349667 saved
查看仓库版本

有了多次的备份以后,生成了不同的版本号,之前我们备份了两次,来看看这个两次的版本号:

restic -r /tmp/backup snapshots

显示的信息如下:

enter password for repository: 
repository 25d29858 opened successfully, password is correct
ID        Time                 Host        Tags        Paths
-----------------------------------------------------------------------------------------
5cc831ec  2022-08-16 16:41:16  lqp                     /home/lqp/manim
37349667  2022-08-16 16:44:07  lqp                     /home/lqp/gotty_linux_amd64.tar.gz
-----------------------------------------------------------------------------------------
2 snapshots

版本还原

要恢复数据出来使用 restore 命令:

restic -r /tmp/backup restore 37349667 --target resticback

这样就把版本号37349667 恢复到resticback 的文件夹了。

enter password for repository: 
repository 25d29858 opened successfully, password is correct
restoring <Snapshot 37349667 of [/home/lqp/gotty_linux_amd64.tar.gz] at 2022-08-16 16:44:07.066697148 +0800 CST by root@lqp> to resticback
root@lqp:/home/lqp# cd resticback/

SFTP 备份还原

本地备份可能还不过瘾,来个 SFTP 备份,为了通过 SFTP 备份数据,您必须首先使用 SSH 设置服务器并让它知道您的公钥。无密码登录很重要,因为如果服务器提示输入凭据,则无法进行自动备份。

配置服务器后,可以通过更改init命令中的 URL 方案来简单地设置 SFTP 存储库:

$ restic -r sftp:user@host:/srv/restic-repo init
enter password for new repository:
enter password again:
created restic repository f1c6108821 at sftp:user@host:/srv/restic-repo
Please note that knowledge of your password is required to access the repository.
Losing your password means that your data is irrecoverably lost.

源码分析

restic采用的是纯go语言编写,对于go语言比较感兴趣的,还有想了解Snapshot原理的可以参考:

package restic

import (
	"context"
	"fmt"
	"os/user"
	"path/filepath"
	"sync"
	"time"

	"golang.org/x/sync/errgroup"

	"github.com/restic/restic/internal/debug"
)

// Snapshot is the state of a resource at one point in time.
type Snapshot struct {
	Time     time.Time `json:"time"`
	Parent   *ID       `json:"parent,omitempty"`
	Tree     *ID       `json:"tree"`
	Paths    []string  `json:"paths"`
	Hostname string    `json:"hostname,omitempty"`
	Username string    `json:"username,omitempty"`
	UID      uint32    `json:"uid,omitempty"`
	GID      uint32    `json:"gid,omitempty"`
	Excludes []string  `json:"excludes,omitempty"`
	Tags     []string  `json:"tags,omitempty"`
	Original *ID       `json:"original,omitempty"`

	id *ID // plaintext ID, used during restore
}

// NewSnapshot returns an initialized snapshot struct for the current user and
// time.
func NewSnapshot(paths []string, tags []string, hostname string, time time.Time) (*Snapshot, error) {
	absPaths := make([]string, 0, len(paths))
	for _, path := range paths {
		p, err := filepath.Abs(path)
		if err == nil {
			absPaths = append(absPaths, p)
		} else {
			absPaths = append(absPaths, path)
		}
	}

	sn := &Snapshot{
		Paths:    absPaths,
		Time:     time,
		Tags:     tags,
		Hostname: hostname,
	}

	err := sn.fillUserInfo()
	if err != nil {
		return nil, err
	}

	return sn, nil
}

// LoadSnapshot loads the snapshot with the id and returns it.
func LoadSnapshot(ctx context.Context, loader LoaderUnpacked, id ID) (*Snapshot, error) {
	sn := &Snapshot{id: &id}
	err := LoadJSONUnpacked(ctx, loader, SnapshotFile, id, sn)
	if err != nil {
		return nil, err
	}

	return sn, nil
}

// SaveSnapshot saves the snapshot sn and returns its ID.
func SaveSnapshot(ctx context.Context, repo SaverUnpacked, sn *Snapshot) (ID, error) {
	return SaveJSONUnpacked(ctx, repo, SnapshotFile, sn)
}

// ForAllSnapshots reads all snapshots in parallel and calls the
// given function. It is guaranteed that the function is not run concurrently.
// If the called function returns an error, this function is cancelled and
// also returns this error.
// If a snapshot ID is in excludeIDs, it will be ignored.
func ForAllSnapshots(ctx context.Context, be Lister, loader LoaderUnpacked, excludeIDs IDSet, fn func(ID, *Snapshot, error) error) error {
	var m sync.Mutex

	// track spawned goroutines using wg, create a new context which is
	// cancelled as soon as an error occurs.
	wg, ctx := errgroup.WithContext(ctx)

	ch := make(chan ID)

	// send list of snapshot files through ch, which is closed afterwards
	wg.Go(func() error {
		defer close(ch)
		return be.List(ctx, SnapshotFile, func(fi FileInfo) error {
			id, err := ParseID(fi.Name)
			if err != nil {
				debug.Log("unable to parse %v as an ID", fi.Name)
				return nil
			}

			if excludeIDs.Has(id) {
				return nil
			}

			select {
			case <-ctx.Done():
				return nil
			case ch <- id:
			}
			return nil
		})
	})

	// a worker receives an snapshot ID from ch, loads the snapshot
	// and runs fn with id, the snapshot and the error
	worker := func() error {
		for id := range ch {
			debug.Log("load snapshot %v", id)
			sn, err := LoadSnapshot(ctx, loader, id)

			m.Lock()
			err = fn(id, sn, err)
			m.Unlock()
			if err != nil {
				return err
			}
		}
		return nil
	}

	// For most snapshots decoding is nearly for free, thus just assume were only limited by IO
	for i := 0; i < int(loader.Connections()); i++ {
		wg.Go(worker)
	}

	return wg.Wait()
}

func (sn Snapshot) String() string {
	return fmt.Sprintf("",
		sn.id.Str(), sn.Paths, sn.Time, sn.Username, sn.Hostname)
}

// ID returns the snapshot's ID.
func (sn Snapshot) ID() *ID {
	return sn.id
}

func (sn *Snapshot) fillUserInfo() error {
	usr, err := user.Current()
	if err != nil {
		return nil
	}
	sn.Username = usr.Username

	// set userid and groupid
	sn.UID, sn.GID, err = uidGidInt(*usr)
	return err
}

// AddTags adds the given tags to the snapshots tags, preventing duplicates.
// It returns true if any changes were made.
func (sn *Snapshot) AddTags(addTags []string) (changed bool) {
nextTag:
	for _, add := range addTags {
		for _, tag := range sn.Tags {
			if tag == add {
				continue nextTag
			}
		}
		sn.Tags = append(sn.Tags, add)
		changed = true
	}
	return
}

// RemoveTags removes the given tags from the snapshots tags and
// returns true if any changes were made.
func (sn *Snapshot) RemoveTags(removeTags []string) (changed bool) {
	for _, remove := range removeTags {
		for i, tag := range sn.Tags {
			if tag == remove {
				// https://github.com/golang/go/wiki/SliceTricks
				sn.Tags[i] = sn.Tags[len(sn.Tags)-1]
				sn.Tags[len(sn.Tags)-1] = ""
				sn.Tags = sn.Tags[:len(sn.Tags)-1]

				changed = true
				break
			}
		}
	}
	return
}

func (sn *Snapshot) hasTag(tag string) bool {
	for _, snTag := range sn.Tags {
		if tag == snTag {
			return true
		}
	}
	return false
}

// HasTags returns true if the snapshot has all the tags in l.
func (sn *Snapshot) HasTags(l []string) bool {
	for _, tag := range l {
		if tag == "" && len(sn.Tags) == 0 {
			return true
		}
		if !sn.hasTag(tag) {
			return false
		}
	}

	return true
}

// HasTagList returns true if either
// - the snapshot satisfies at least one TagList, so there is a TagList in l
//   for which all tags are included in sn, or
// - l is empty
func (sn *Snapshot) HasTagList(l []TagList) bool {
	debug.Log("testing snapshot with tags %v against list: %v", sn.Tags, l)

	if len(l) == 0 {
		return true
	}

	for _, tags := range l {
		if sn.HasTags(tags) {
			debug.Log("  snapshot satisfies %v %v", tags, l)
			return true
		}
	}

	return false
}

func (sn *Snapshot) hasPath(path string) bool {
	for _, snPath := range sn.Paths {
		if path == snPath {
			return true
		}
	}
	return false
}

// HasPaths returns true if the snapshot has all of the paths.
func (sn *Snapshot) HasPaths(paths []string) bool {
	for _, path := range paths {
		if !sn.hasPath(path) {
			return false
		}
	}

	return true
}

// HasHostname returns true if either
// - the snapshot hostname is in the list of the given hostnames, or
// - the list of given hostnames is empty
func (sn *Snapshot) HasHostname(hostnames []string) bool {
	if len(hostnames) == 0 {
		return true
	}

	for _, hostname := range hostnames {
		if sn.Hostname == hostname {
			return true
		}
	}

	return false
}

// Snapshots is a list of snapshots.
type Snapshots []*Snapshot

// Len returns the number of snapshots in sn.
func (sn Snapshots) Len() int {
	return len(sn)
}

// Less returns true iff the ith snapshot has been made after the jth.
func (sn Snapshots) Less(i, j int) bool {
	return sn[i].Time.After(sn[j].Time)
}

// Swap exchanges the two snapshots.
func (sn Snapshots) Swap(i, j int) {
	sn[i], sn[j] = sn[j], sn[i]
}

你可能感兴趣的:(Github每日精选,github,debian,linux)