Practice - Azure File Storage

Azure Storage 是微软 Azure 云提供的云端存储解决方案，当前支持的存储类型有 Blob、Queue、File 和 Table。

azure-storage.png

File Storage 是什么？

Azure File Storage 是一个通过 Server Message Block (SMB) 协议提供云端文件共享的服务。简单来说，可以把它当作NAS存储。

Azure File Storage的结构

image

• 存储帐户：对 Azure 存储服务的所有访问都要通过存储帐户来完成。

• 共享：文件存储共享是 Azure 中的 SMB 文件共享。 所有目录和文件都必须在父共享中创建。 一个帐户可以包含无限数量的共享，一个共享可以存储无限数量的文件，直到达到文件共享的 5TiB 总容量为止。

• 目录：可选的目录层次结构。

• 文件：共享中的文件。 文件大小最大可以为 1 TiB。

• URL 格式：对于使用文件 REST 协议提出的 Azure 文件共享请求，可采用以下 URL 格式对文件进行寻址：

    https://.file.core.windows.net///directories>/

真实的例子：

https://royhkfile.file.core.windows.net/backup/tools/HDP-localRepo.md

实践操作 (通过网页）

创建存储帐号

create storage account

创建 Storage File

storage-file.png

创建 Share, directory

file-share.png

file-dir.png

查看如何mount

在Share这一级中查看

mount option

从 Linux VM 验证

登录同一个region 的VM，创建一个目录用来挂载 File share

$ sudo mkdir /roymount

获取网页中得到的mount 命令如下：

sudo mount -t cifs //royhkfile.file.core.windows.net/backup [mount point] -o vers=3.0,username=royhkfile,password=RWM6QpGYRzb8omcYaZzAlKksACmKdnmyPB5tmCTx3wSLBPAgn8bqJU66s+mCHBQoY8zRP22Z3mkLrLVY+1BavA==,dir_mode=0777,file_mode=0777,sec=ntlmssp

替换掉[mount piont]部分,执行命令

$ sudo mount -t cifs //royhkfile.file.core.windows.net/backup /roymount -o vers=3.0,username=royhkfile,password=RWM6QpGYRzb8omcYaZzAlKksACmKdnmyPB5tmCTx3wSLBPAgn8bqJU66s+mCHBQoY8zRP22Z3mkLrLVY+1BavA==,dir_mode=0777,file_mode=0777,sec=ntlmssp

或者不从网页查看，直接替换下面范例的参数后，执行命令。

sudo mount -t cifs //.file.core.windows.net/  -o vers=,username=,password=,dir_mode=0777,file_mode=0777,serverino

查看内容

$ tree /roymount
/roymount
└── tools
    └── HDP-localRepo.md

证实 Azure File 已经在Linux VM mount 成功。

实践操作 (通过Azure CLI）

Reference document

Create resource group

$ az group create -n roy-ea-rg -l eastasia

Create storage account

$ az storage account create -g roy-ea-rg -n roystore -l eastasia --sku Standard_LRS
{
  "accessTier": null,
  "creationTime": "2018-09-14T05:51:11.646609+00:00",
  "customDomain": null,
  "enableHttpsTrafficOnly": false,
  "encryption": {
    "keySource": "Microsoft.Storage",
    "keyVaultProperties": null,
    "services": {
      "blob": {
        "enabled": true,
        "lastEnabledTime": "2018-09-14T05:51:11.724732+00:00"
      },
      "file": {
        "enabled": true,
        "lastEnabledTime": "2018-09-14T05:51:11.724732+00:00"
      },
      "queue": null,
      "table": null
    }
  },
  "id": "/subscriptions/xxx/resourceGroups/roy-ea-rg/providers/Microsoft.Storage/storageAccounts/roystore",
  "identity": null,
  "kind": "Storage",
  "lastGeoFailoverTime": null,
  "location": "eastasia",
  "name": "roystore",
  "networkRuleSet": {
    "bypass": "AzureServices",
    "defaultAction": "Allow",
    "ipRules": [],
    "virtualNetworkRules": []
  },
  "primaryEndpoints": {
    "blob": "https://roystore.blob.core.windows.net/",
    "file": "https://roystore.file.core.windows.net/",
    "queue": "https://roystore.queue.core.windows.net/",
    "table": "https://roystore.table.core.windows.net/"
  },
  "primaryLocation": "eastasia",
  "provisioningState": "Succeeded",
  "resourceGroup": "roy-ea-rg",
  "secondaryEndpoints": null,
  "secondaryLocation": null,
  "sku": {
    "capabilities": null,
    "kind": null,
    "locations": null,
    "name": "Standard_LRS",
    "resourceType": null,
    "restrictions": null,
    "tier": "Standard"
  },
  "statusOfPrimary": "available",
  "statusOfSecondary": null,
  "tags": {},
  "type": "Microsoft.Storage/storageAccounts"
}

Get the storage account key

$ az storage account keys list -g roy-ea-rg --account-name roystore --query "[0].value" | tr -d '"'
fNmS+T9WvWgfPeGW4SuqShZ8vHrkkaVEp2g8Pla7rG7hBee7zRbUWgzNHA0xtfjQy6xsyGiL3JrXWlpauBNoKg==
$ STORAGEKEY=$(az storage account keys list -g roy-ea-rg --account-name roystore --query "[0].value" | tr -d '"')

Create an Azure file share

Now, you can create your first Azure file share. Create file shares by using the az storage share create command. This example creates an Azure file share named myshare:

$ az storage share create --account-name roystore \
> --account-key $STORAGEKEY \
> --name "myshare"
{
  "created": true
}

Create a directory

$ az storage directory create \
> --account-name roystore \
> --account-key $STORAGEKEY \
> --share-name "myshare" \
> --name "myDirectory"
{
  "created": true
}

Upload a file

$ az storage file upload \
    --account-name $STORAGEACCT \
    --account-key $STORAGEKEY \
    --share-name "myshare" \
    --source "main.yml" \
    --path "myDirectory/main.yml"
Finished[#############################################################]  100.0000%

Verify

$ az storage file list \
    --account-name $STORAGEACCT \
    --account-key $STORAGEKEY \
    --share-name "myshare" \
    --path "myDirectory" \
    --output table
Name        Content Length  Type    Last Modified
--------  ----------------  ------  ---------------
main.yml              1238  file

Download a file

$ az storage file download \
    --account-name $STORAGEACCT \
    --account-key $STORAGEKEY \
    --share-name "myshare" \
    --path "myDirectory/main.yml" \
    --dest "/tmp/main.yml"
Finished[#############################################################]  100.0000%
{
  "content": null,
  "metadata": {},
  "name": "main.yml",
  "properties": {
    "contentLength": 1238,
    "contentRange": "bytes 0-1237/1238",
    "contentSettings": {
      "cacheControl": null,
      "contentDisposition": null,
      "contentEncoding": null,
      "contentLanguage": null,
      "contentType": "application/octet-stream"
    },
    "copy": {
      "completionTime": null,
      "id": null,
      "progress": null,
      "source": null,
      "status": null,
      "statusDescription": null
    },
    "etag": "\"0x8D61A0936E26AA0\"",
    "lastModified": "2018-09-14T06:13:39+00:00",
    "serverEncrypted": true
  }
}
$ ll /tmp/main.yml
-rw-r--r--  1 royzeng  wheel   1.2K Sep 14 14:26 /tmp/main.yml

Mount the share

sudo mount -t cifs //$STORAGEACCT.file.core.windows.net/myshare /mnt/MyAzureFileShare -o vers=3.0,username=$STORAGEACCT,password=$STORAGEKEY,dir_mode=0777,file_mode=0777,serverino

e.g.

ubuntu@roy-ea-vm-k8s-01:~$ sudo mount -t cifs //roystore.file.core.windows.net/myshare /royfile -o vers=3.0,username=roystore,password=fNmS+T9WvWgfPeGW4SuqShZ8vHrkkaVEp2g8Pla7rG7hBee7zRbUWgzNHA0xtfjQy6xsyGiL3JrXWlpauBNoKg==,dir_mode=0777,file_mode=0777,serverino
ubuntu@roy-ea-vm-k8s-01:~$
ubuntu@roy-ea-vm-k8s-01:~$ df -hP /royfile
Filesystem                                Size  Used Avail Use% Mounted on
//roystore.file.core.windows.net/myshare  5.0T   64K  5.0T   1% /royfile
ubuntu@roy-ea-vm-k8s-01:~$ cd /royfile
ubuntu@roy-ea-vm-k8s-01:/royfile$ ls
myDirectory
ubuntu@roy-ea-vm-k8s-01:/royfile$ ls myDirectory
main.yml
ubuntu@roy-ea-vm-k8s-01:/royfile$ mkdir testdir
ubuntu@roy-ea-vm-k8s-01:/royfile$ cd testdir
ubuntu@roy-ea-vm-k8s-01:/royfile/testdir$ echo aaa>newfile
ubuntu@roy-ea-vm-k8s-01:/royfile/testdir$ cat newfile
aaa

Persist the mount

update /etc/fstab

//myaccountname.file.core.windows.net/mystorageshare /mnt/mymountpoint cifs vers=3.0,username=mystorageaccount,password=myStorageAccountKeyEndingIn==,dir_mode=0777,file_mode=0777

or increased security in production environments, you should store your credentials outside of fstab.