webassembly010 ggml的其他封装:Python,Rust ,javascript

其他封装
Tensor library for machine learning, inspired by ggml	https://github.com/candrewlee14/zgml
Bleeding edge low level Rust binding for GGML	https://github.com/KerfuffleV2/ggml-sys-bleedingedge
Your customized AI characters - personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2.	https://github.com/nrl-ai/CustomChar
cpu inference example for mpt-7b with ggml+rust	https://github.com/apage43/mptgen
Python bindings for the GGML GPT-J Laguage model	https://github.com/abdeladim-s/pygptj
Run any ggml ported ML models directly on your web browser with ggml.js	https://github.com/rahuldshetty/ggml.js
Python bindings for the Transformer models implemented in C/C++ using GGML library.	https://github.com/marella/ctransformers

ggml-python

• 在github上有一个使用python对ggml进行封装的项目ggml-python，通过下面的例子可以看出使用ggml进行推理主要包括以下几个步骤：
• 上下文环境创建=>
• tensors数据初始化=>
• 构建计算图=>
• 设置tensor值=>
• 前向推理=>
• 输出值，释放上下文<=>

#  https://github.com/abetlen/ggml-python
import ggml
import ctypes

# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params=params)

# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)

# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)

gf = ggml.ggml_build_forward(f)

# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)

# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, ctypes.pointer(gf), 1)

# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0

# Free the context
ggml.ggml_free(ctx)

CG

• https://ggml-python.readthedocs.io/en/latest/
• https://github.com/ggerganov/ggml/tree/master/examples/python
• https://github.com/ggerganov/whisper.cpp/issues/9
• https://github.com/go-skynet/LocalAI