aws dynamodb 使用dynamodb local在本地运行nosql测试环境

参考资料

• DynamoDB Local使用

• https://docs.aws.amazon.com/zh_cn/amazondynamodb/latest/developerguide/DynamoDBLocal.html

• https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS.html

• https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/globals.html

在没有aws账号或者仅仅在本地进行测试时，无法与dynamodb服务通信使得很多测试无法进行。实际上dynamodb为我们提供了本地运行环境。

在本地搭建dynamodb的好处在于

• 节省预置的吞吐量、数据存储和数据传输费用
• 无需 Internet 连接

我们在之前的文章中已经涉及到很多此类工具，例如

• ecs本地元数据mock
• apigateway和lambda本地环境
• ec2元数据mock

dynamodb的local环境有三种部署方式

• maven依赖项
• 单独jar包
• docker容器

我们尝试下后两种方式

jar包部署

前提条件

• 具有 Java 运行时环境 (JRE) 版本 8.x 或更高版本

启动后，默认监听8000端口，在cli使用参数 –endpoint-url指定本地终端节点

java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -sharedDb
# java -Djava.library.path=./DynamoDBLocal_lib -jar DynamoDBLocal.jar -inMemory

关于启动命令参数的含义

• -sharedDb， dynamodb创建名为shared-local-instance.db数据库文件，删除此文件会丢失所有数据。查看文件类型实际上是个sqlite数据库文件

  $ file shared-local-instance.db
  shared-local-instance.db: SQLite 3.x database
  

  省略参数则数据文件为myaccesskeyid_region.db

• -inMemory，不会编写任何数据库文件，将数据写入内存中

• -optimizeDbBeforeStartup

• 始终使用billingModeSummary，忽略预置的吞吐量设置

存在的一些限制

• 事务上限为 25 个项目

• Scan 操作将按顺序执行。不支持并行扫描

• 表数据的读取和写入操作的速度仅受计算机速度的限制

• 读取操作具有最终一致性

• 每个结果集返回的数据有 1 MB 的限制，但是查询索引时会计算整个项目的大小

• 不存在表分区活动

• 不会为事务 API 引发 TransactionConflictExceptions

• 本地表名称不区分大小写（dynamodb服务会区分）

我们还是按照之前cli的步骤测试

aws dynamodb create-table \
    --table-name Music \
    --attribute-definitions \
        AttributeName=Artist,AttributeType=S \
        AttributeName=SongTitle,AttributeType=S \
    --key-schema \
        AttributeName=Artist,KeyType=HASH \
        AttributeName=SongTitle,KeyType=RANGE \
    --provisioned-throughput \
        ReadCapacityUnits=5,WriteCapacityUnits=5 \
    --table-class STANDARD \
    --endpoint-url http://127.0.0.1:8000

查看表，账号为000000000000

aws dynamodb describe-table --table-name Music --endpoint-url http://127.0.0.1:8000
{
    "Table": {
        "AttributeDefinitions": [
            {
                "AttributeName": "Artist",
                "AttributeType": "S"
            },
            {
                "AttributeName": "SongTitle",
                "AttributeType": "S"
            }
        ],
        "TableName": "Music",
        "KeySchema": [
            {
                "AttributeName": "Artist",
                "KeyType": "HASH"
            },
            {
                "AttributeName": "SongTitle",
                "KeyType": "RANGE"
            }
        ],
        "TableStatus": "ACTIVE",
        "CreationDateTime": "2023-03-11T14:05:12.895000+00:00",
        "ProvisionedThroughput": {
            "LastIncreaseDateTime": "1970-01-01T00:00:00+00:00",
            "LastDecreaseDateTime": "1970-01-01T00:00:00+00:00",
            "NumberOfDecreasesToday": 0,
            "ReadCapacityUnits": 5,
            "WriteCapacityUnits": 5
        },
        "TableSizeBytes": 0,
        "ItemCount": 0,
        "TableArn": "arn:aws:dynamodb:ddblocal:000000000000:table/Music"
    }
}

使用docker部署

创建docker-compose.yml文件

version: '3.8'
services:
  dynamodb-local:
    command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
    image: "public.ecr.aws/aws-dynamodb-local/aws-dynamodb-local:latest"
    container_name: dynamodb-local
    ports:
      - "8000:8000"
    volumes:
      - "./docker/dynamodb:/home/dynamodblocal/data"
    working_dir: /home/dynamodblocal

也可以将容器和dynamodb-local部署在一个ns中

version: '3.8'
services:
  dynamodb-local:
    command: "-jar DynamoDBLocal.jar -sharedDb -dbPath ./data"
    image: "public.ecr.aws/aws-dynamodb-local/aws-dynamodb-local:latest"
    container_name: dynamodb-local
    ports:
      - "8000:8000"
    volumes:
      - "./docker/dynamodb:/home/dynamodblocal/data"
    working_dir: /home/dynamodblocal
  app-node:
    depends_on:
      - dynamodb-local
    image: public.ecr.aws/aws-cli/aws-cli:2.11.2
    container_name: app-node
    links:
      - "dynamodb-local"
    ports:
     - "8080:8080"
    environment:
      AWS_ACCESS_KEY_ID: 'DUMMYIDEXAMPLE'
      AWS_SECRET_ACCESS_KEY: 'DUMMYEXAMPLEKEY'
      REGION: 'cn-north-1'
    command:
      dynamodb describe-limits --endpoint-url http://dynamodb-local:8000

启动查看日志

[+] Running 5/5
 ⠿ dynamodb-local Pulled                                                                                                                      16.3s
   ⠿ 1e78b99dd1fd Pull complete                                                                                                                7.2s
   ⠿ 6aabd9041a5c Pull complete                                                                                                               11.0s
   ⠿ 4f4fb700ef54 Pull complete                                                                                                               11.0s
   ⠿ 601cf5909367 Pull complete                                                                                                               11.8s
[+] Running 3/2
 ⠿ Network local-dynamodb_default  Created                                                                                                     0.1s
 ⠿ Container dynamodb-local        Created                                                                                                     1.4s
 ⠿ Container app-node              Created                                                                                                     0.0s
Attaching to app-node, dynamodb-local
dynamodb-local  | Initializing DynamoDB Local with the following configuration:
dynamodb-local  | Port: 8000
dynamodb-local  | InMemory:     false
dynamodb-local  | DbPath:       ./data
dynamodb-local  | SharedDb:     true
dynamodb-local  | shouldDelayTransientStatuses: false
dynamodb-local  | CorsParams:   null
dynamodb-local  |
app-node        | {
app-node        |     "AccountMaxReadCapacityUnits": 80000,
app-node        |     "AccountMaxWriteCapacityUnits": 80000,
app-node        |     "TableMaxReadCapacityUnits": 40000,
app-node        |     "TableMaxWriteCapacityUnits": 40000
app-node        | }
app-node exited with code 0

问题不大，测试环境响应速度快，效率高，测试和开发方面有优势