【如何训练一个中译英翻译器】LSTM机器翻译模型部署之ncnn（python）（五）

系列文章
【如何训练一个中译英翻译器】LSTM机器翻译seq2seq字符编码（一）
【如何训练一个中译英翻译器】LSTM机器翻译模型训练与保存（二）
【如何训练一个中译英翻译器】LSTM机器翻译模型部署（三）
【如何训练一个中译英翻译器】LSTM机器翻译模型部署之onnx（python）（四）

一、事情准备

这篇是在【如何训练一个中译英翻译器】LSTM机器翻译模型部署之onnx（python）（四）的基础上进行的，要用到文件为：

input_words.txt
target_words.txt
config.json
encoder_model-sim.onnx
decoder_model-sim.onnx

其中的onnx就是用来转为ncnn模型的，这里借助了onnx这个中间商，所以前面我们需要先通过onnxsim对模型进行simplify，要不然在模型转换时会出现op不支持的情况（模型转换不仅有中间商这个例子，目前还可以通过pnnx直接将pytorch模型转为ncnn，感兴趣的小伙伴可以去折腾下）
老规矩，先给出工具：

onnx2ncnn：https://github.com/Tencent/ncnn
netron：https://netron.app

二、模型转换

这里进行onnx转ncnn，通过命令进行转换

onnx2ncnn onnxModel/encoder_model-sim.onnx ncnnModel/encoder_model.param ncnnModel/encoder_model.bin
onnx2ncnn onnxModel/decoder_model-sim.onnx ncnnModel/decoder_model.param ncnnModel/decoder_model.bin

转换成功可以看到：

转换之后可以对模型进行优化，但是奇怪的是，这里优化了不起作用，去不了MemoryData这些没用的op

ncnnoptimize ncnnModel/encoder_model.param ncnnModel/encoder_model.bin ncnnModel/encoder_model.param ncnnModel/encoder_model.bin 1
ncnnoptimize ncnnModel/decoder_model.param ncnnModel/decoder_model.bin ncnnModel/decoder_model.param ncnnModel/decoder_model.bin 1

三、ncnn模型加载与推理（python版）

跟onnx的推理比较类似，就是函数的调用方法有点不同，这里先用python实现，验证下是否没问题，方面后面部署到其它端，比如android。
主要包括：模型加载、推理模型搭建跟模型推理，但要注意的是这里的输入输出名称需要在param这个文件里面获取。

采用netron分别查看encoder与decoder的网络结构，获取输入输出名称:

encoder:
输入输出分别如图

decoder:

输入

输出：

有点问题，先把调试代码贴在下面了

import numpy as np
import ncnn


# 加载字符
# 从 input_words.txt 文件中读取字符串
with open('config/input_words.txt', 'r') as f:
    input_words = f.readlines()
    input_characters = [line.rstrip('\n') for line in input_words]

# 从 target_words.txt 文件中读取字符串
with open('config/target_words.txt', 'r', newline='') as f:
    target_words = [line.strip() for line in f.readlines()]
    target_characters = [char.replace('\\t', '\t').replace('\\n', '\n') for char in target_words]

#字符处理，以方便进行编码
input_token_index = dict([(char, i) for i, char in enumerate(input_characters)])
target_token_index = dict([(char, i) for i, char in enumerate(target_characters)])

# something readable.
reverse_input_char_index = dict(
    (i, char) for char, i in input_token_index.items())
reverse_target_char_index = dict(
    (i, char) for char, i in target_token_index.items())
num_encoder_tokens = len(input_characters) # 英文字符数量
num_decoder_tokens = len(target_characters) # 中文文字数量

import json
with open('config/config.json', 'r') as file:
    loaded_data = json.load(file)

# 从加载的数据中获取max_encoder_seq_length和max_decoder_seq_length的值
max_encoder_seq_length = loaded_data["max_encoder_seq_length"]
max_decoder_seq_length = loaded_data["max_decoder_seq_length"]


encoder_model = ncnn.Net()

encoder_model.load_param("ncnnModel/encoder_model.param")
encoder_model.load_model("ncnnModel/encoder_model.bin")

decoder_model = ncnn.Net()
decoder_model.load_param("ncnnModel/decoder_model.param")
decoder_model.load_model("ncnnModel/decoder_model.bin")


def decode_sequence(input_seq):
    # Encode the input as state vectors.
    ex_encoder = encoder_model.create_extractor()
    ex_encoder.input("input_1", ncnn.Mat(input_seq))
    _, LSTM_1 = ex_encoder.extract("LSTM__31:1")
    _, LSTM_2 = ex_encoder.extract("LSTM__31:2")
    
    print(LSTM_1)
    print(LSTM_2)
    


    # Generate empty target sequence of length 1.
    target_seq = np.zeros((1, 1, 849))
    # Populate the first character of target sequence with the start character.
    target_seq[0, 0, target_token_index['\t']] = 1.
    # this target_seq you can treat as initial state

    # Sampling loop for a batch of sequences
    # (to simplify, here we assume a batch of size 1).
    stop_condition = False
    decoded_sentence = ''
    while not stop_condition:
        ex_decoder = decoder_model.create_extractor()
        
        print(ncnn.Mat(target_seq))
        
        print("---------")

        
        
        
        ex_decoder.input("input_2", ncnn.Mat(target_seq))
        ex_decoder.input("input_3", LSTM_1)
        ex_decoder.input("input_4", LSTM_2)
        _, output_tokens = ex_decoder.extract("dense")
        _, h = ex_decoder.extract("lstm_1")
        _, c = ex_decoder.extract("lstm_1_1")
	

        
        print(output_tokens)
        
        print(h)
        print(c)
        
        print(fdsf)
        
        output_tokens = np.array(output_tokens)
        h = np.array(h)
        c = np.array(c)
        print(output_tokens.shape)
        print(output_tokens.shape)
        print(h.shape)
        print(c.shape)
        #print(gfdgd)
        
        
	    
    
        #output_tokens, h, c = decoder_model.predict([target_seq] + states_value)

        # Sample a token
        # argmax: Returns the indices of the maximum values along an axis
        # just like find the most possible char
        sampled_token_index = np.argmax(output_tokens[0, -1, :])
        # find char using index
        sampled_char = reverse_target_char_index[sampled_token_index]
        # and append sentence
        decoded_sentence += sampled_char

        # Exit condition: either hit max length
        # or find stop character.
        if (sampled_char == '\n' or len(decoded_sentence) > max_decoder_seq_length):
            stop_condition = True

        # Update the target sequence (of length 1).
        # append then ?
        # creating another new target_seq
        # and this time assume sampled_token_index to 1.0
        target_seq = np.zeros((1, 1, num_decoder_tokens))
        target_seq[0, 0, sampled_token_index] = 1.

        # Update states
        # update states, frome the front parts
        states_value = [h, c]

    return decoded_sentence
    
    
import numpy as np

input_text = "Call me."
encoder_input_data = np.zeros(
    (1,max_encoder_seq_length, num_encoder_tokens),
    dtype='float32')
for t, char in enumerate(input_text):
    print(char)
    # 3D vector only z-index has char its value equals 1.0
    encoder_input_data[0,t, input_token_index[char]] = 1.


input_seq = encoder_input_data
decoded_sentence = decode_sequence(input_seq)
print('-')
print('Input sentence:', input_text)
print('Decoded sentence:', decoded_sentence)