webassembly010 ggml的其他封装:Python,Rust ,javascript

其他封装
Tensor library for machine learning, inspired by ggml https://github.com/candrewlee14/zgml
Bleeding edge low level Rust binding for GGML https://github.com/KerfuffleV2/ggml-sys-bleedingedge
Your customized AI characters - personal assistants on any hardware! With llama.cpp, whisper.cpp, ggml, LLaMA-v2. https://github.com/nrl-ai/CustomChar
cpu inference example for mpt-7b with ggml+rust https://github.com/apage43/mptgen
Python bindings for the GGML GPT-J Laguage model https://github.com/abdeladim-s/pygptj
Run any ggml ported ML models directly on your web browser with ggml.js https://github.com/rahuldshetty/ggml.js
Python bindings for the Transformer models implemented in C/C++ using GGML library. https://github.com/marella/ctransformers

ggml-python

  • 在github上有一个使用python对ggml进行封装的项目ggml-python,通过下面的例子可以看出使用ggml进行推理主要包括以下几个步骤:
  • 上下文环境创建=>
  • tensors数据初始化=>
  • 构建计算图=>
  • 设置tensor值=>
  • 前向推理=>
  • 输出值,释放上下文<=>
#  https://github.com/abetlen/ggml-python
import ggml
import ctypes

# Allocate a new context with 16 MB of memory
params = ggml.ggml_init_params(mem_size=16 * 1024 * 1024, mem_buffer=None)
ctx = ggml.ggml_init(params=params)

# Instantiate tensors
x = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
a = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)
b = ggml.ggml_new_tensor_1d(ctx, ggml.GGML_TYPE_F32, 1)

# Use ggml operations to build a computational graph
x2 = ggml.ggml_mul(ctx, x, x)
f = ggml.ggml_add(ctx, ggml.ggml_mul(ctx, a, x2), b)

gf = ggml.ggml_build_forward(f)

# Set the input values
ggml.ggml_set_f32(x, 2.0)
ggml.ggml_set_f32(a, 3.0)
ggml.ggml_set_f32(b, 4.0)

# Compute the graph
ggml.ggml_graph_compute_with_ctx(ctx, ctypes.pointer(gf), 1)

# Get the output value
output = ggml.ggml_get_f32_1d(f, 0)
assert output == 16.0

# Free the context
ggml.ggml_free(ctx)

CG

  • https://ggml-python.readthedocs.io/en/latest/
  • https://github.com/ggerganov/ggml/tree/master/examples/python
  • https://github.com/ggerganov/whisper.cpp/issues/9
  • https://github.com/go-skynet/LocalAI

你可能感兴趣的:(移动端,python,rust,javascript,wasm)