Yongqiang Cheng

MNIST Examples for GGML - Fully connected network

1. Build
2. MNIST Examples for GGML
- 2.1. Obtaining the data
- 2.2. Fully connected network
- - 2.2.1. To train a fully connected model in PyTorch and save it as a GGUF file
  - 2.2.2. To evaluate the model on the CPU using GGML
  - 2.2.3. To train a fully connected model on the CPU using GGML
- 2.3. Hardware Acceleration
References

1. Build

https://github.com/ggml-org/ggml

git clone https://github.com/ggml-org/ggml
cd ggml

# install python dependencies in a virtual environment
python3.10 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# build the examples
mkdir build && cd build
cmake ..
cmake --build . --config Release -j 8

(base) yongqiang@yongqiang:~$ cd llm_work/
(base) yongqiang@yongqiang:~/llm_work$ mkdir ggml_25_02_15
(base) yongqiang@yongqiang:~/llm_work$ cd ggml_25_02_15/
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15$ git clone https://github.com/ggml-org/ggml.git
Cloning into 'ggml'...
remote: Enumerating objects: 13755, done.
remote: Counting objects: 100% (498/498), done.
remote: Compressing objects: 100% (193/193), done.
remote: Total 13755 (delta 331), reused 335 (delta 303), pack-reused 13257 (from 3)
Receiving objects: 100% (13755/13755), 12.88 MiB | 213.00 KiB/s, done.
Resolving deltas: 100% (9411/9411), done.
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15$ cd ggml/
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ pip install -r requirements.txt

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ vim build_ggml_linux_cpu.sh
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ chmod a+x build_ggml_linux_cpu.sh
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ cat build_ggml_linux_cpu.sh
#! /bin/bash

# build the examples
mkdir build && cd build
cmake ..
cmake --build . --config Debug -j 8

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ bash build_ggml_linux_cpu.sh
-- The C compiler identification is GNU 9.4.0
-- The CXX compiler identification is GNU 9.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native
-- x86 detected
-- Linux detected
-- Configuring done (10.1s)
-- Generating done (0.1s)
-- Build files have been written to: /home/yongqiang/llm_work/ggml_25_02_15/ggml/build
...
[ 99%] Linking CXX executable ../../bin/gpt-j
[ 99%] Built target gpt-j
[100%] Linking CXX executable ../../bin/sam
[100%] Built target sam
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$

2. MNIST Examples for GGML

https://github.com/ggml-org/ggml/tree/master/examples/mnist

This directory contains simple examples of how to use GGML for training and inference using the MNIST dataset. All commands listed in this README assume the working directory to be examples/mnist.

MNIST dataset
https://yann.lecun.com/exdb/mnist/

Please note that training in GGML is a work-in-progress and not production ready.

2.1. Obtaining the data

A description of the dataset can be found on Yann LeCun’s website.

While it is also in principle possible to download the dataset from this website these downloads are frequently throttled and
it is recommended to use HuggingFace instead.

The dataset will be downloaded automatically when running mnist-train-fc.py.

2.2. Fully connected network

For our first example we will train a fully connected network.

2.2.1. To train a fully connected model in PyTorch and save it as a GGUF file

$ python3 mnist-train-fc.py mnist-fc-f32.gguf

...

Test loss: 0.066377+-0.010468, Test accuracy: 97.94+-0.14%

Model tensors saved to mnist-fc-f32.gguf:
fc1.weight       (500, 784)
fc1.bias         (500,)
fc2.weight       (10, 500)
fc2.bias         (10,)

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml$ cd examples/mnist/
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$ python3 mnist-train-fc.py mnist-fc-f32.gguf
Downloading http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz
Failed to download (trying next):
HTTP Error 404: Not Found

Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz
Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-images-idx3-ubyte.gz to ./data/MNIST/raw/train-images-idx3-ubyte.gz
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9.91M/9.91M [00:11<00:00, 864kB/s]
Extracting ./data/MNIST/raw/train-images-idx3-ubyte.gz to ./data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz
Failed to download (trying next):
HTTP Error 404: Not Found

Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz
Downloading https://ossci-datasets.s3.amazonaws.com/mnist/train-labels-idx1-ubyte.gz to ./data/MNIST/raw/train-labels-idx1-ubyte.gz
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 28.9k/28.9k [00:00<00:00, 57.4kB/s]
Extracting ./data/MNIST/raw/train-labels-idx1-ubyte.gz to ./data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz
Failed to download (trying next):
HTTP Error 404: Not Found

Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz
Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw/t10k-images-idx3-ubyte.gz
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1.65M/1.65M [00:07<00:00, 224kB/s]
Extracting ./data/MNIST/raw/t10k-images-idx3-ubyte.gz to ./data/MNIST/raw

Downloading http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz
Failed to download (trying next):
HTTP Error 404: Not Found

Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz
Downloading https://ossci-datasets.s3.amazonaws.com/mnist/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4.54k/4.54k [00:00<00:00, 3.66MB/s]
Extracting ./data/MNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/MNIST/raw

Epoch [01/30], Step [10000/60000], Loss: 1.7476, Accuracy: 61.69%
Epoch [01/30], Step [20000/60000], Loss: 1.2862, Accuracy: 71.74%
Epoch [01/30], Step [30000/60000], Loss: 1.0215, Accuracy: 76.76%
Epoch [01/30], Step [40000/60000], Loss: 0.8656, Accuracy: 79.79%
Epoch [01/30], Step [50000/60000], Loss: 0.7615, Accuracy: 81.86%
Epoch [01/30], Step [60000/60000], Loss: 0.6898, Accuracy: 83.28%
Epoch [02/30], Step [10000/60000], Loss: 0.2972, Accuracy: 91.74%
Epoch [02/30], Step [20000/60000], Loss: 0.2857, Accuracy: 91.98%
Epoch [02/30], Step [30000/60000], Loss: 0.2836, Accuracy: 91.99%
Epoch [02/30], Step [40000/60000], Loss: 0.2780, Accuracy: 92.10%
Epoch [02/30], Step [50000/60000], Loss: 0.2750, Accuracy: 92.20%
Epoch [02/30], Step [60000/60000], Loss: 0.2693, Accuracy: 92.41%
Epoch [03/30], Step [10000/60000], Loss: 0.2285, Accuracy: 93.94%
Epoch [03/30], Step [20000/60000], Loss: 0.2247, Accuracy: 93.82%
Epoch [03/30], Step [30000/60000], Loss: 0.2141, Accuracy: 94.13%
Epoch [03/30], Step [40000/60000], Loss: 0.2140, Accuracy: 94.14%
Epoch [03/30], Step [50000/60000], Loss: 0.2101, Accuracy: 94.17%
Epoch [03/30], Step [60000/60000], Loss: 0.2071, Accuracy: 94.21%
Epoch [04/30], Step [10000/60000], Loss: 0.1752, Accuracy: 94.96%
Epoch [04/30], Step [20000/60000], Loss: 0.1781, Accuracy: 94.88%
Epoch [04/30], Step [30000/60000], Loss: 0.1715, Accuracy: 95.07%
Epoch [04/30], Step [40000/60000], Loss: 0.1702, Accuracy: 95.15%
Epoch [04/30], Step [50000/60000], Loss: 0.1695, Accuracy: 95.19%
Epoch [04/30], Step [60000/60000], Loss: 0.1672, Accuracy: 95.27%
Epoch [05/30], Step [10000/60000], Loss: 0.1426, Accuracy: 96.04%
Epoch [05/30], Step [20000/60000], Loss: 0.1413, Accuracy: 95.99%
Epoch [05/30], Step [30000/60000], Loss: 0.1417, Accuracy: 96.01%
Epoch [05/30], Step [40000/60000], Loss: 0.1425, Accuracy: 95.94%
Epoch [05/30], Step [50000/60000], Loss: 0.1410, Accuracy: 95.98%
Epoch [05/30], Step [60000/60000], Loss: 0.1395, Accuracy: 96.03%
Epoch [06/30], Step [10000/60000], Loss: 0.1234, Accuracy: 96.65%
Epoch [06/30], Step [20000/60000], Loss: 0.1188, Accuracy: 96.71%
Epoch [06/30], Step [30000/60000], Loss: 0.1184, Accuracy: 96.71%
Epoch [06/30], Step [40000/60000], Loss: 0.1169, Accuracy: 96.77%
Epoch [06/30], Step [50000/60000], Loss: 0.1163, Accuracy: 96.78%
Epoch [06/30], Step [60000/60000], Loss: 0.1167, Accuracy: 96.75%
Epoch [07/30], Step [10000/60000], Loss: 0.1041, Accuracy: 97.19%
Epoch [07/30], Step [20000/60000], Loss: 0.1018, Accuracy: 97.28%
Epoch [07/30], Step [30000/60000], Loss: 0.1006, Accuracy: 97.29%
Epoch [07/30], Step [40000/60000], Loss: 0.1017, Accuracy: 97.18%
Epoch [07/30], Step [50000/60000], Loss: 0.1014, Accuracy: 97.19%
Epoch [07/30], Step [60000/60000], Loss: 0.1008, Accuracy: 97.20%
Epoch [08/30], Step [10000/60000], Loss: 0.0929, Accuracy: 97.49%
Epoch [08/30], Step [20000/60000], Loss: 0.0906, Accuracy: 97.53%
Epoch [08/30], Step [30000/60000], Loss: 0.0886, Accuracy: 97.61%
Epoch [08/30], Step [40000/60000], Loss: 0.0897, Accuracy: 97.53%
Epoch [08/30], Step [50000/60000], Loss: 0.0888, Accuracy: 97.56%
Epoch [08/30], Step [60000/60000], Loss: 0.0882, Accuracy: 97.57%
Epoch [09/30], Step [10000/60000], Loss: 0.0721, Accuracy: 98.05%
Epoch [09/30], Step [20000/60000], Loss: 0.0733, Accuracy: 98.03%
Epoch [09/30], Step [30000/60000], Loss: 0.0745, Accuracy: 98.01%
Epoch [09/30], Step [40000/60000], Loss: 0.0762, Accuracy: 97.92%
Epoch [09/30], Step [50000/60000], Loss: 0.0770, Accuracy: 97.90%
Epoch [09/30], Step [60000/60000], Loss: 0.0765, Accuracy: 97.91%
Epoch [10/30], Step [10000/60000], Loss: 0.0676, Accuracy: 98.22%
Epoch [10/30], Step [20000/60000], Loss: 0.0681, Accuracy: 98.22%
Epoch [10/30], Step [30000/60000], Loss: 0.0663, Accuracy: 98.25%
Epoch [10/30], Step [40000/60000], Loss: 0.0663, Accuracy: 98.24%
Epoch [10/30], Step [50000/60000], Loss: 0.0680, Accuracy: 98.18%
Epoch [10/30], Step [60000/60000], Loss: 0.0677, Accuracy: 98.18%
Epoch [11/30], Step [10000/60000], Loss: 0.0584, Accuracy: 98.50%
Epoch [11/30], Step [20000/60000], Loss: 0.0584, Accuracy: 98.51%
Epoch [11/30], Step [30000/60000], Loss: 0.0598, Accuracy: 98.45%
Epoch [11/30], Step [40000/60000], Loss: 0.0605, Accuracy: 98.43%
Epoch [11/30], Step [50000/60000], Loss: 0.0606, Accuracy: 98.40%
Epoch [11/30], Step [60000/60000], Loss: 0.0608, Accuracy: 98.39%
Epoch [12/30], Step [10000/60000], Loss: 0.0576, Accuracy: 98.54%
Epoch [12/30], Step [20000/60000], Loss: 0.0551, Accuracy: 98.61%
Epoch [12/30], Step [30000/60000], Loss: 0.0546, Accuracy: 98.59%
Epoch [12/30], Step [40000/60000], Loss: 0.0544, Accuracy: 98.59%
Epoch [12/30], Step [50000/60000], Loss: 0.0536, Accuracy: 98.57%
Epoch [12/30], Step [60000/60000], Loss: 0.0540, Accuracy: 98.54%
Epoch [13/30], Step [10000/60000], Loss: 0.0482, Accuracy: 98.62%
Epoch [13/30], Step [20000/60000], Loss: 0.0487, Accuracy: 98.69%
Epoch [13/30], Step [30000/60000], Loss: 0.0486, Accuracy: 98.73%
Epoch [13/30], Step [40000/60000], Loss: 0.0492, Accuracy: 98.72%
Epoch [13/30], Step [50000/60000], Loss: 0.0484, Accuracy: 98.73%
Epoch [13/30], Step [60000/60000], Loss: 0.0485, Accuracy: 98.72%
Epoch [14/30], Step [10000/60000], Loss: 0.0416, Accuracy: 98.91%
Epoch [14/30], Step [20000/60000], Loss: 0.0428, Accuracy: 98.84%
Epoch [14/30], Step [30000/60000], Loss: 0.0437, Accuracy: 98.85%
Epoch [14/30], Step [40000/60000], Loss: 0.0433, Accuracy: 98.88%
Epoch [14/30], Step [50000/60000], Loss: 0.0433, Accuracy: 98.87%
Epoch [14/30], Step [60000/60000], Loss: 0.0435, Accuracy: 98.86%
Epoch [15/30], Step [10000/60000], Loss: 0.0366, Accuracy: 99.02%
Epoch [15/30], Step [20000/60000], Loss: 0.0376, Accuracy: 99.03%
Epoch [15/30], Step [30000/60000], Loss: 0.0389, Accuracy: 99.01%
Epoch [15/30], Step [40000/60000], Loss: 0.0386, Accuracy: 99.02%
Epoch [15/30], Step [50000/60000], Loss: 0.0381, Accuracy: 99.05%
Epoch [15/30], Step [60000/60000], Loss: 0.0391, Accuracy: 99.00%
Epoch [16/30], Step [10000/60000], Loss: 0.0372, Accuracy: 99.10%
Epoch [16/30], Step [20000/60000], Loss: 0.0336, Accuracy: 99.19%
Epoch [16/30], Step [30000/60000], Loss: 0.0323, Accuracy: 99.23%
Epoch [16/30], Step [40000/60000], Loss: 0.0338, Accuracy: 99.17%
Epoch [16/30], Step [50000/60000], Loss: 0.0340, Accuracy: 99.17%
Epoch [16/30], Step [60000/60000], Loss: 0.0350, Accuracy: 99.12%
Epoch [17/30], Step [10000/60000], Loss: 0.0313, Accuracy: 99.30%
Epoch [17/30], Step [20000/60000], Loss: 0.0308, Accuracy: 99.33%
Epoch [17/30], Step [30000/60000], Loss: 0.0305, Accuracy: 99.29%
Epoch [17/30], Step [40000/60000], Loss: 0.0311, Accuracy: 99.28%
Epoch [17/30], Step [50000/60000], Loss: 0.0317, Accuracy: 99.27%
Epoch [17/30], Step [60000/60000], Loss: 0.0318, Accuracy: 99.24%
Epoch [18/30], Step [10000/60000], Loss: 0.0281, Accuracy: 99.37%
Epoch [18/30], Step [20000/60000], Loss: 0.0286, Accuracy: 99.36%
Epoch [18/30], Step [30000/60000], Loss: 0.0291, Accuracy: 99.35%
Epoch [18/30], Step [40000/60000], Loss: 0.0291, Accuracy: 99.34%
Epoch [18/30], Step [50000/60000], Loss: 0.0289, Accuracy: 99.35%
Epoch [18/30], Step [60000/60000], Loss: 0.0288, Accuracy: 99.35%
Epoch [19/30], Step [10000/60000], Loss: 0.0245, Accuracy: 99.47%
Epoch [19/30], Step [20000/60000], Loss: 0.0246, Accuracy: 99.46%
Epoch [19/30], Step [30000/60000], Loss: 0.0248, Accuracy: 99.43%
Epoch [19/30], Step [40000/60000], Loss: 0.0250, Accuracy: 99.41%
Epoch [19/30], Step [50000/60000], Loss: 0.0250, Accuracy: 99.43%
Epoch [19/30], Step [60000/60000], Loss: 0.0253, Accuracy: 99.43%
Epoch [20/30], Step [10000/60000], Loss: 0.0216, Accuracy: 99.55%
Epoch [20/30], Step [20000/60000], Loss: 0.0222, Accuracy: 99.59%
Epoch [20/30], Step [30000/60000], Loss: 0.0225, Accuracy: 99.56%
Epoch [20/30], Step [40000/60000], Loss: 0.0228, Accuracy: 99.54%
Epoch [20/30], Step [50000/60000], Loss: 0.0228, Accuracy: 99.55%
Epoch [20/30], Step [60000/60000], Loss: 0.0233, Accuracy: 99.52%
Epoch [21/30], Step [10000/60000], Loss: 0.0211, Accuracy: 99.61%
Epoch [21/30], Step [20000/60000], Loss: 0.0196, Accuracy: 99.62%
Epoch [21/30], Step [30000/60000], Loss: 0.0203, Accuracy: 99.62%
Epoch [21/30], Step [40000/60000], Loss: 0.0202, Accuracy: 99.62%
Epoch [21/30], Step [50000/60000], Loss: 0.0205, Accuracy: 99.60%
Epoch [21/30], Step [60000/60000], Loss: 0.0211, Accuracy: 99.58%
Epoch [22/30], Step [10000/60000], Loss: 0.0178, Accuracy: 99.72%
Epoch [22/30], Step [20000/60000], Loss: 0.0191, Accuracy: 99.68%
Epoch [22/30], Step [30000/60000], Loss: 0.0194, Accuracy: 99.66%
Epoch [22/30], Step [40000/60000], Loss: 0.0192, Accuracy: 99.66%
Epoch [22/30], Step [50000/60000], Loss: 0.0193, Accuracy: 99.64%
Epoch [22/30], Step [60000/60000], Loss: 0.0189, Accuracy: 99.65%
Epoch [23/30], Step [10000/60000], Loss: 0.0167, Accuracy: 99.67%
Epoch [23/30], Step [20000/60000], Loss: 0.0168, Accuracy: 99.69%
Epoch [23/30], Step [30000/60000], Loss: 0.0164, Accuracy: 99.72%
Epoch [23/30], Step [40000/60000], Loss: 0.0165, Accuracy: 99.72%
Epoch [23/30], Step [50000/60000], Loss: 0.0165, Accuracy: 99.72%
Epoch [23/30], Step [60000/60000], Loss: 0.0170, Accuracy: 99.70%
Epoch [24/30], Step [10000/60000], Loss: 0.0141, Accuracy: 99.84%
Epoch [24/30], Step [20000/60000], Loss: 0.0142, Accuracy: 99.83%
Epoch [24/30], Step [30000/60000], Loss: 0.0144, Accuracy: 99.81%
Epoch [24/30], Step [40000/60000], Loss: 0.0152, Accuracy: 99.78%
Epoch [24/30], Step [50000/60000], Loss: 0.0154, Accuracy: 99.77%
Epoch [24/30], Step [60000/60000], Loss: 0.0154, Accuracy: 99.77%
Epoch [25/30], Step [10000/60000], Loss: 0.0136, Accuracy: 99.85%
Epoch [25/30], Step [20000/60000], Loss: 0.0141, Accuracy: 99.81%
Epoch [25/30], Step [30000/60000], Loss: 0.0140, Accuracy: 99.79%
Epoch [25/30], Step [40000/60000], Loss: 0.0138, Accuracy: 99.79%
Epoch [25/30], Step [50000/60000], Loss: 0.0139, Accuracy: 99.79%
Epoch [25/30], Step [60000/60000], Loss: 0.0141, Accuracy: 99.77%
Epoch [26/30], Step [10000/60000], Loss: 0.0116, Accuracy: 99.85%
Epoch [26/30], Step [20000/60000], Loss: 0.0117, Accuracy: 99.85%
Epoch [26/30], Step [30000/60000], Loss: 0.0126, Accuracy: 99.82%
Epoch [26/30], Step [40000/60000], Loss: 0.0129, Accuracy: 99.82%
Epoch [26/30], Step [50000/60000], Loss: 0.0127, Accuracy: 99.82%
Epoch [26/30], Step [60000/60000], Loss: 0.0126, Accuracy: 99.82%
Epoch [27/30], Step [10000/60000], Loss: 0.0103, Accuracy: 99.90%
Epoch [27/30], Step [20000/60000], Loss: 0.0115, Accuracy: 99.89%
Epoch [27/30], Step [30000/60000], Loss: 0.0117, Accuracy: 99.88%
Epoch [27/30], Step [40000/60000], Loss: 0.0118, Accuracy: 99.86%
Epoch [27/30], Step [50000/60000], Loss: 0.0114, Accuracy: 99.88%
Epoch [27/30], Step [60000/60000], Loss: 0.0114, Accuracy: 99.88%
Epoch [28/30], Step [10000/60000], Loss: 0.0102, Accuracy: 99.95%
Epoch [28/30], Step [20000/60000], Loss: 0.0097, Accuracy: 99.91%
Epoch [28/30], Step [30000/60000], Loss: 0.0099, Accuracy: 99.89%
Epoch [28/30], Step [40000/60000], Loss: 0.0100, Accuracy: 99.90%
Epoch [28/30], Step [50000/60000], Loss: 0.0104, Accuracy: 99.89%
Epoch [28/30], Step [60000/60000], Loss: 0.0106, Accuracy: 99.88%
Epoch [29/30], Step [10000/60000], Loss: 0.0091, Accuracy: 99.93%
Epoch [29/30], Step [20000/60000], Loss: 0.0095, Accuracy: 99.92%
Epoch [29/30], Step [30000/60000], Loss: 0.0093, Accuracy: 99.92%
Epoch [29/30], Step [40000/60000], Loss: 0.0095, Accuracy: 99.92%
Epoch [29/30], Step [50000/60000], Loss: 0.0094, Accuracy: 99.92%
Epoch [29/30], Step [60000/60000], Loss: 0.0095, Accuracy: 99.91%
Epoch [30/30], Step [10000/60000], Loss: 0.0083, Accuracy: 99.91%
Epoch [30/30], Step [20000/60000], Loss: 0.0082, Accuracy: 99.92%
Epoch [30/30], Step [30000/60000], Loss: 0.0078, Accuracy: 99.94%
Epoch [30/30], Step [40000/60000], Loss: 0.0080, Accuracy: 99.94%
Epoch [30/30], Step [50000/60000], Loss: 0.0083, Accuracy: 99.93%
Epoch [30/30], Step [60000/60000], Loss: 0.0085, Accuracy: 99.92%

Training took 51.71s

Test loss: 0.065348+-0.011133, Test accuracy: 98.07+-0.14%

Model tensors saved to mnist-fc-f32.gguf:
fc1.weight       (500, 784)
fc1.bias         (500,)
fc2.weight       (10, 500)
fc2.bias         (10,)
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$

The training script includes an evaluation of the model on the test set.

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$ ls -l data/MNIST/raw/
total 65008
-rw-r--r-- 1 yongqiang yongqiang  7840016 Feb 15 23:25 t10k-images-idx3-ubyte
-rw-r--r-- 1 yongqiang yongqiang  1648877 Feb 15 23:25 t10k-images-idx3-ubyte.gz
-rw-r--r-- 1 yongqiang yongqiang    10008 Feb 15 23:25 t10k-labels-idx1-ubyte
-rw-r--r-- 1 yongqiang yongqiang     4542 Feb 15 23:25 t10k-labels-idx1-ubyte.gz
-rw-r--r-- 1 yongqiang yongqiang 47040016 Feb 15 23:25 train-images-idx3-ubyte
-rw-r--r-- 1 yongqiang yongqiang  9912422 Feb 15 23:25 train-images-idx3-ubyte.gz
-rw-r--r-- 1 yongqiang yongqiang    60008 Feb 15 23:25 train-labels-idx1-ubyte
-rw-r--r-- 1 yongqiang yongqiang    28881 Feb 15 23:25 train-labels-idx1-ubyte.gz
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$

2.2.2. To evaluate the model on the CPU using GGML

$ ../../build/bin/mnist-eval mnist-fc-f32.gguf data/MNIST/raw/t10k-images-idx3-ubyte data/MNIST/raw/t10k-labels-idx1-ubyte

________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
__________________________________####__________________
______________________________########__________________
__________________________##########____________________
______________________##############____________________
____________________######________####__________________
__________________________________####__________________
__________________________________####__________________
________________________________####____________________
______________________________####______________________
________________________##########______________________
______________________########__####____________________
________________________##__________##__________________
____________________________________##__________________
__________________________________##____________________
__________________________________##____________________
________________________________##______________________
____________________________####________________________
__________##____________######__________________________
__________##############________________________________
________________####____________________________________
________________________________________________________
________________________________________________________
________________________________________________________
________________________________________________________
ggml_cuda_init: GGML_CUDA_FORCE_MMQ:    no
ggml_cuda_init: GGML_CUDA_FORCE_CUBLAS: no
ggml_cuda_init: found 1 CUDA devices:
  Device 0: NVIDIA GeForce RTX 3090, compute capability 8.6, VMM: yes
mnist_model: using CUDA0 (NVIDIA GeForce RTX 3090) as primary backend
mnist_model: unsupported operations will be executed on the following fallback backends (in order of priority):
mnist_model:  - CPU (AMD Ryzen 9 5950X 16-Core Processor)
mnist_model_init_from_file: loading model weights from 'mnist-fc-f32.gguf'
mnist_model_init_from_file: model arch is mnist-fc
mnist_model_init_from_file: successfully loaded weights from mnist-fc-f32.gguf
main: loaded model in 109.44 ms
mnist_model_eval: model evaluation on 10000 images took 76.92 ms, 7.69 us/image
main: predicted digit is 3
main: test_loss=0.066379+-0.009101
main: test_acc=97.94+-0.14%

In addition to the evaluation on the test set the GGML evaluation also prints a random image from the test set as well as the model prediction for said image.

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$ ../../build/bin/mnist-eval mnist-fc-f32.gguf data/MNIST/raw/t10k-images-idx3-ubyte data/MNIST/raw/t10k-labels-idx1-ubyte

...

mnist_model: using CPU (Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz) as primary backend
mnist_model_init_from_file: loading model weights from 'mnist-fc-f32.gguf'
mnist_model_init_from_file: model arch is mnist-fc
mnist_model_init_from_file: successfully loaded weights from mnist-fc-f32.gguf
main: loaded model in 3.84 ms
mnist_model_eval: model evaluation on 10000 images took 83.78 ms, 8.38 us/image
main: predicted digit is 1
main: test_loss=0.065348+-0.009093
main: test_acc=98.07+-0.14%
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$

2.2.3. To train a fully connected model on the CPU using GGML

$ ../../build/bin/mnist-train mnist-fc mnist-fc-f32.gguf data/MNIST/raw/train-images-idx3-ubyte data/MNIST/raw/train-labels-idx1-ubyte

(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$ ../../build/bin/mnist-train mnist-fc mnist-fc-f32.gguf data/MNIST/raw/train-images-idx3-ubyte data/MNIST/raw/train-labels-idx1-ubyte
mnist_model: using CPU (Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz) as primary backend
mnist_model_init_random: initializing random weights for a fully connected model
ggml_opt_fit: epoch 0001/0030:
train: [=========================| data=057000/057000, loss=0.842470+-0.057215, accuracy=81.99+-0.16%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.354736+-0.009396, accuracy=89.77+-0.55%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0002/0030:
train: [=========================| data=057000/057000, loss=0.297789+-0.003739, accuracy=91.54+-0.12%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.267287+-0.011581, accuracy=92.37+-0.48%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0003/0030:
train: [=========================| data=057000/057000, loss=0.236971+-0.003106, accuracy=93.32+-0.10%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.221299+-0.010660, accuracy=93.80+-0.44%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0004/0030:
train: [=========================| data=057000/057000, loss=0.195029+-0.002957, accuracy=94.46+-0.10%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.187524+-0.010246, accuracy=94.50+-0.42%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0005/0030:
train: [=========================| data=057000/057000, loss=0.163346+-0.002484, accuracy=95.41+-0.09%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.167672+-0.008361, accuracy=95.00+-0.40%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0006/0030:
train: [=========================| data=057000/057000, loss=0.140522+-0.002344, accuracy=96.04+-0.08%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.150562+-0.010338, accuracy=95.60+-0.37%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0007/0030:
train: [=========================| data=057000/057000, loss=0.122155+-0.002059, accuracy=96.52+-0.08%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.137456+-0.008758, accuracy=96.43+-0.34%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0008/0030:
train: [=========================| data=057000/057000, loss=0.105244+-0.001992, accuracy=97.05+-0.07%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.126794+-0.008748, accuracy=96.43+-0.34%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0009/0030:
train: [=========================| data=057000/057000, loss=0.094218+-0.001796, accuracy=97.39+-0.07%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.116145+-0.006691, accuracy=96.77+-0.32%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0010/0030:
train: [=========================| data=057000/057000, loss=0.084340+-0.001764, accuracy=97.62+-0.06%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.111088+-0.006398, accuracy=96.80+-0.32%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0011/0030:
train: [=========================| data=057000/057000, loss=0.074447+-0.001579, accuracy=97.94+-0.06%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.104478+-0.008579, accuracy=97.13+-0.30%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0012/0030:
train: [=========================| data=057000/057000, loss=0.066881+-0.001480, accuracy=98.12+-0.06%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.098624+-0.006870, accuracy=97.13+-0.30%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0013/0030:
train: [=========================| data=057000/057000, loss=0.059336+-0.001320, accuracy=98.38+-0.05%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.097907+-0.006335, accuracy=97.37+-0.29%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0014/0030:
train: [=========================| data=057000/057000, loss=0.053577+-0.001345, accuracy=98.58+-0.05%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.096755+-0.007440, accuracy=97.33+-0.29%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0015/0030:
train: [=========================| data=057000/057000, loss=0.048068+-0.001061, accuracy=98.72+-0.05%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.090680+-0.007853, accuracy=97.60+-0.28%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0016/0030:
train: [=========================| data=057000/057000, loss=0.043474+-0.001046, accuracy=98.87+-0.04%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.092639+-0.007943, accuracy=97.37+-0.29%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0017/0030:
train: [=========================| data=057000/057000, loss=0.039780+-0.000824, accuracy=98.95+-0.04%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.082859+-0.006335, accuracy=97.73+-0.27%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0018/0030:
train: [=========================| data=057000/057000, loss=0.035487+-0.001002, accuracy=99.12+-0.04%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.081594+-0.007036, accuracy=97.87+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0019/0030:
train: [=========================| data=057000/057000, loss=0.032300+-0.000904, accuracy=99.23+-0.04%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.082995+-0.007232, accuracy=97.73+-0.27%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0020/0030:
train: [=========================| data=057000/057000, loss=0.028769+-0.000809, accuracy=99.35+-0.03%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.076938+-0.006239, accuracy=98.03+-0.25%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0021/0030:
train: [=========================| data=057000/057000, loss=0.026851+-0.000771, accuracy=99.39+-0.03%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.078551+-0.006827, accuracy=97.83+-0.27%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0022/0030:
train: [=========================| data=057000/057000, loss=0.024355+-0.000553, accuracy=99.48+-0.03%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.077560+-0.007714, accuracy=97.87+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0023/0030:
train: [=========================| data=057000/057000, loss=0.021858+-0.000612, accuracy=99.54+-0.03%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.077453+-0.007612, accuracy=97.97+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0024/0030:
train: [=========================| data=057000/057000, loss=0.019889+-0.000548, accuracy=99.62+-0.03%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.075201+-0.006989, accuracy=97.97+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0025/0030:
train: [=========================| data=057000/057000, loss=0.017892+-0.000533, accuracy=99.72+-0.02%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.075809+-0.007840, accuracy=97.90+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0026/0030:
train: [=========================| data=057000/057000, loss=0.016237+-0.000444, accuracy=99.75+-0.02%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.078347+-0.008378, accuracy=97.97+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0027/0030:
train: [=========================| data=057000/057000, loss=0.015027+-0.000438, accuracy=99.78+-0.02%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.075985+-0.007250, accuracy=97.93+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0028/0030:
train: [=========================| data=057000/057000, loss=0.014067+-0.000477, accuracy=99.80+-0.02%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.076615+-0.008804, accuracy=98.07+-0.25%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0029/0030:
train: [=========================| data=057000/057000, loss=0.012649+-0.000385, accuracy=99.85+-0.02%, t=00:00:01, ETA=00:00:00]
val:   [=========================| data=003000/003000, loss=0.076876+-0.009151, accuracy=97.93+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: epoch 0030/0030:
train: [=========================| data=057000/057000, loss=0.011452+-0.000376, accuracy=99.86+-0.02%, t=00:00:01, ETA=00:00:00]]
val:   [=========================| data=003000/003000, loss=0.076046+-0.008447, accuracy=98.00+-0.26%, t=00:00:00, ETA=00:00:00]

ggml_opt_fit: training took 00:00:34
mnist_model_save: saving model to 'mnist-fc-f32.gguf'
(base) yongqiang@yongqiang:~/llm_work/ggml_25_02_15/ggml/examples/mnist$

It can then be evaluated with the same binary as above.

2.3. Hardware Acceleration

Both the training and evaluation code is agnostic in terms of hardware as long as the corresponding GGML backend has implemented the necessary operations.

A specific backend can be selected by appending the above commands with a backend name.

The compute graphs then schedule the operations to preferentially use the specified backend.

Note that if a backend does not implement some of the necessary operations a CPU fallback is used instead which may result in bad performance.

References

[1] Yongqiang Cheng, https://yongqiang.blog.csdn.net/

你可能感兴趣的:(ggml,-,llama.cpp,-,whisper.cpp,GGML,MNIST,Examples,Fully,connected)

rabbitmq 的错误记录 lisanmengmeng 系统运维架构消息队列 rabbitmq 分布式
集群添加：rabbitmqctljoin_clusterrabbit@1错误提示：Clusteringnoderabbit@[email protected]:unabletoconnecttonodes[rabbit@1]:nodedownDIAGNOSTICS===========attemptedtocontact:[rabbit@1]rabbit@1:*connectedtoepm
PAT A 1003 Emergency cwn_ 算法数据结构图论 c++
Asanemergencyrescueteamleaderofacity,youaregivenaspecialmapofyourcountry.Themapshowsseveralscatteredcitiesconnectedbysomeroads.Amountofrescueteamsineachcityandthelengthofeachroadbetweenanypairofcities
Learning Fully Convolutional Networks for Iterative Non-blind Deconvolution论文阅读青铜锁00 #退化论文阅读深度学习论文阅读图像处理
LearningFullyConvolutionalNetworksforIterativeNon-blindDeconvolution1.研究目标与实际问题1.1研究目标1.2实际意义2.创新方法与模型设计2.1核心框架：迭代式梯度域处理2.1.1模型架构2.2关键技术实现2.2.1梯度域去噪网络2.2.2解卷积模块（核心公式实现）2.2.3损失函数设计2.2.4超参数端到端学习2.3与传统方法
PyTorch 中 nn.Linear() 参数详解与实战解析（gpt）草莓奶忻深度学习 pytorch gpt 人工智能
PyTorch中nn.Linear()参数详解与实战解析在使用PyTorch构建神经网络时，nn.Linear()是最常用也最基础的模块之一。它用于实现一个全连接层（FullyConnectedLayer），本质上就是对输入进行一次线性变换：y=xAT+by=xA^T+by=xAT+b本文将详细介绍nn.Linear()的参数含义、属性说明、初始化机制，并结合实际代码案例帮助你真正理解它的工作原理
本地运行大型语言模型(LLM)的实践指南 yunwu12777 语言模型人工智能自然语言处理
技术背景介绍近年来，项目如llama.cpp、Ollama、GPT4All等的流行标志着在本地设备上运行大型语言模型（LLM）的需求日益增长。选择在本地运行LLM，至少有两个重要的好处：隐私和成本。隐私上，数据不需要发送到第三方，避免了商业服务条款的限制；成本方面，无需支付推理费用，尤其是对于那些需要大量计算的应用，如长时间的模拟和总结。核心原理解析在本地运行LLM，需要准备以下几个条件：开源LL
Python网安-zip文件暴力破解（仅供学习） Whoisshutiao python网安 python 开发语言网络安全
目录源码在这里需要的模块准备一个密码本和需要破解的ZIP文件一行一行地从密码文件中读取每个密码。核心部分注意，需要修改上段代码注释里的这段具有编码问题的代码：源码在这里https://github.com/Wist-fully/Attack/tree/cracker需要的模块fromtqdmimporttqdmimportzipfileimportpyzipper准备一个密码本和需要破解的ZIP文
Python网安-ftp服务暴力破解（仅供学习） Whoisshutiao python 网络安全开发语言
目录源码在这里需要导入的模块连接ftp，并设置密码本和线程核心代码设置线程源码在这里https://github.com/Wist-fully/Attack/tree/cracker需要导入的模块importftplibfromthreadingimportThreadimportqueue连接ftp，并设置密码本和线程host="192.168.6.6"user="student"port=21
Python实例题：基于 KNN 算法的手写数字识别
目录Python实例题题目要求：解题思路：代码实现：Python实例题题目基于KNN算法的手写数字识别要求：实现一个基于K-NearestNeighbors(KNN)算法的手写数字识别系统。支持以下功能：使用MNIST数据集训练和测试模型实现KNN分类算法可视化手写数字样本评估模型性能（准确率、混淆矩阵等）添加用户交互界面，允许用户绘制数字并进行识别。解题思路：使用sklearn加载MNIST数据
单片机_no target connected问题解决方法一条破秋裤单片机问题总结单片机嵌入式硬件 stm32
问题01---notargetconnected解决方法此问题是在烧录呼吸灯程序时产生，可能因定时器配置错误导致。（定时器，PWM）
Densenet模型花卉图像分类深度学习乐园分类数据挖掘人工智能
项目源码获取方式见文章末尾！600多个深度学习项目资料，快来加入社群一起学习吧。《------往期经典推荐------》项目名称1.【基于CNN-RNN的影像报告生成】2.【卫星图像道路检测DeepLabV3Plus模型】3.【GAN模型实现二次元头像生成】4.【CNN模型实现mnist手写数字识别】5.【fasterRCNN模型实现飞机类目标检测】6.【CNN-LSTM住宅用电量预测】7.【VG
微服务架构设计模式资源下载介绍：掌握微服务设计精髓，助力架构升级
微服务架构设计模式资源下载介绍：掌握微服务设计精髓，助力架构升级【下载地址】微服务架构设计模式资源下载介绍探索微服务架构的奥秘，掌握设计模式的精髓。本仓库提供了一本权威的英文书籍《MicroservicePatterns:WithexamplesinJava》的PDF资源，由克里斯-理查森精心撰写。书中不仅涵盖了微服务的基本概念，还深入探讨了服务拆分、服务发现、负载均衡等关键主题，辅以丰富的实例和
Dpdk环境的搭建 Flying Fish（HHH） dpdk 分布式计算机网络
1、下载源码：gitclonehttps://github.com/DPDK/dpdk2、编译命令CC=gccmeson-Dlibdir=lib--default-library=static-Dexamples=allx86_64-native-linuxapp-gcc-Dbuildtype=debugoptimizedninja-Cx86_64-native-linuxapp-gcc-j110
python打卡训练营Day41 珂宝_ python打卡训练营 python
importnumpyasnpfromtensorflowimportkerasfromtensorflow.kerasimportlayers#加载和预处理数据(x_train,y_train),(x_test,y_test)=keras.datasets.mnist.load_data()x_train=x_train.reshape(-1,28,28,1).astype("float32")
基于PyTorch的MNIST手写数字识别（配置手写板使用）热心不起来的市民小周 CV 项目实操 pytorch 人工智能 python
基于PyTorch的MNIST手写数字识别（配置手写板使用）代码详见：https://github.com/xiaozhou-alt/CNN_MNIST文章目录基于PyTorch的MNIST手写数字识别（配置手写板使用）一、项目介绍二、数据集介绍三、项目实现1.环境准备2.项目文件夹结构3.数据预处理4.开始训练！(1)数据加载(2)数据转换(3)模型定义(4)训练过程(5)评估测试四、结果展示一
多头注意力机制中全连接函数不知更鸟深度学习
在神经网络（特别是Transformer中的多头注意力机制）中，全连接函数（FullyConnectedLayer,FCLayer）通常指的是一个线性变换层，即nn.Linear在PyTorch中的实现。它本质上是一个矩阵乘法加上偏置（bias）的操作，用于对输入数据进行线性变换。1.全连接函数（nn.Linear）是什么？nn.Linear(d_model,d_model)表示一个全连接层，它的
【vLLM 学习】Disaggregated Prefill
vLLM是一款专为大语言模型推理加速而设计的框架，实现了KV缓存内存几乎零浪费，解决了内存管理瓶颈问题。更多vLLM中文文档及教程可访问→https://vllm.hyper.ai/*在线运行vLLM入门教程：零基础分步指南源码examples/offline_inference/disaggregated_prefill.py#SPDX-License-Identifier:Apache-2.0
以numpy或Torch的格式存储的公开数据集以科技求富强多模态聚类学习数据库 python 大数据 database
现有的以numpy或Torch的格式存储的公开数据集1.**MNIST**2.**CIFAR-10/CIFAR-100**3.**ImageNet**4.**COCO(CommonObjectsinContext)**5.**PascalVOC**6.**Fashion-MNIST**7.**BostonHousing**8.**Iris**9.**KITTI**10.**CelebA**11.*
Vue Macros 开源项目教程班妲盼Joyce
VueMacros开源项目教程vue-macrosExploreandextendmoremacrosandsyntaxsugartoVue.项目地址:https://gitcode.com/gh_mirrors/vu/vue-macros项目的目录结构及介绍VueMacros项目的目录结构如下：vue-macros/├──docs/├──examples/├──packages/│├──api-
深度学习--一个分类的例子惊讶的猫人工智能深度学习机器学习
说明：本文会实现自定义模型实现对MINIST数据集的训练，训练完之后还会使用测试集进行测试。所依托的训练集和测试集都是由datasets.MNIST获取到的。步骤下载数据现在来介绍加载MNIST数据集并准备训练和测试数据的逻辑。数据预处理：transforms.Compose创建了一个数据预处理管道，将多个数据转换操作组合在一起。将图像转换为PyTorch的张量（tensor).对图像进行标准化，
Rust 编写与 Scala 类似功能的实例 KENYCHEN奉孝 Rust rust 后端 scala 开发语言
Rust中运行3D游戏引擎示例以下是在Rust中运行3D游戏引擎示例游戏的方法和相关资源：使用Bevy引擎Bevy是一个轻量级、模块化的Rust游戏引擎，适合快速构建3D游戏示例。安装后可通过官方示例库运行3D场景：cargoinstallbevygitclonehttps://github.com/bevyengine/bevycdbevy/examples/3dcargorun--exampl
【动手学深度学习】4.2~4.3 多层感知机的实现 XiaoJ1234567 《动手学深度学习》深度学习人工智能 MLP 多层感知机
目录4.2.多层感知机的从零开始实现1）初始化模型参数2）激活函数3）模型4）损失函数5）训练4.3.多层感知机的简洁实现1）模型2）小结.4.2.多层感知机的从零开始实现现在让我们实现一个多层感知机。为了与之前softmax回归获得的结果进行比较，我们将继续使用Fashion-MNIST图像分类数据集。importtorchfromtorchimportnnfromd2limporttorcha
Python str.format() 函数在 LLM prompt 生成中的使用（微调、刷库等） ctrl A_ctrl C_ctrl V #大模型llm python prompt 开发语言自然语言处理
str.format()函数介绍：Pythonformat格式化函数，用于增强字符串格式化。在LLM中批量生成prompt的简单示例：template="我现在有一个用户问题和系统的答案,帮我把答案中和用户问题最直接的关键词提取出来。\n问题：“{query}”\n答案：“{answer}”"examples={'query':"秒换装教学",'answer':"在王者荣耀游戏中，秒换装的技巧主要
Python实现简单的深度学习实践 master_chenchengg python python Python python开发 IT
Python实现简单的深度学习实践Python：通往深度学习世界的钥匙动手搭建你的第一个神经网络模型从零开始，用Python解析MNIST手写数字识别超越基础：使用Keras快速构建卷积神经网络实战演练：训练一个简单的图像分类器Python：通往深度学习世界的钥匙在当今这个数据驱动的时代，Python无疑成为了打开深度学习大门的金钥匙。它不仅语法简洁、易于上手，而且拥有强大的社区支持和丰富的库资源
tortoise-orm在fastapi的lifespan中访问数据库表时报for r in self._routers:TypeError: ‘NoneType‘ object is not it waketzheng fastapi python
解决：升级tortoise-orm到0.21.1以上版本，或使用github上的待发布版poetryadd'tortoise-orm>=0.21.1'||poetryadd'git+https://github.com/tortoise/tortoise-orm.git'或直接pip安装https://pip.pypa.io/en/stable/cli/pip_install/#examplesp
《强连通分量(tarjan算法)》基础概念
文章目录一、算法概述二、算法思路三、伪代码实现1.类定义与数据结构2.主程序示例四、算法解释1.初始化阶段2.DFS遍历与时间戳更新3.强连通分量识别4.示例演示五、复杂度分析一、算法概述定义：Tarjan算法是一种用于在有向图中求解强连通分量（StronglyConnectedComponent,SCC）的高效算法。强连通分量指有向图中任意两顶点互相可达的最大子图。核心思想：基于深度优先搜索（D
linux启动lsyncd服务,lsyncd实时同步工具王小约 linux启动lsyncd服务
lsyncd实时同步工具：Lsyncd实际上是lua语言封装了inotify和rsync工具，采用了Linux内核(2.6.13及以后)里的inotify触发机制，然后通过rsync去差异同步，达到实时的效果。安装：apt-getinstalllsyncd配置：mkdir/etc/lsyncdcp-rv/usr/share/doc/lsyncd/examples/lrsync.lua/etc/ls
前端 flex弹性布局之弹性布局中文换行默默地写代码 html flex 布局前端 html5 flex布局
阮一峰布局语法：http://www.ruanyifeng.com/blog/2015/07/flex-grammar.htmlFlex布局实例篇：http://www.ruanyifeng.com/blog/2015/07/flex-examples.html我是第一66666在此不讲flex布局语法跟实例，请参考以上阮一峰老师的教程。主要提醒大家注意：在使用弹性布局的时候，如果div内部写的是
FPGA基础 -- Verilog 结构建模之未连接的端口 sz66cm FPGA基础 fpga开发
Verilog中结构建模时未连接的端口（UnconnectedPorts），包括：什么是未连接端口如何显式地忽略端口连接实际使用场景工具综合与仿真中的注意事项未连接端口的工程规范建议一、什么是“未连接的端口”？当你例化一个模块时，如果某个端口并不需要使用（例如该模块的调试接口、保留接口、未启用通道），你可以选择不连接这个端口。✅二、未连接端口的写法1.命名连接.port()空写法（推荐）my_mo
深度学习“炼丹”实战：用LeNet驯服MNIST“神兽” AI妈妈手把手深度学习人工智能 LeNet cnn 模型训练学习笔记 MNIST
宝子们，在深度学习的神秘世界里，咱们就像一群“炼丹师”，而模型就是咱们精心炼制的“丹药”，数据集则是炼丹的“原材料”。今天，咱们就用经典的LeNet卷积神经网络模型，在MNIST手写数字数据集这个“原材料宝库”里，炼制出一颗能精准识别数字的“神奇丹药”！LeNet网络结构回顾，见：深度学习图像分类六大经典网络结构全解析一、MNIST数据集：炼丹的“珍贵原料”MNIST数据集可是深度学习界的“老牌明
visual studio小番茄插件某些快捷键失效 0点51胜 visual studio ide
问题Alt+O切换头文件和源文件失效。背景最近升级了visualstudio，多了一些插件原因Alt+O快捷键被其他插件占用了解决方案工具→选项→环境→键盘搜索这个VAssistX.OpenCorrespondingFile（切换头/源文件）发现命令的快捷键是空的，给他分配快捷键发现被Edit.IntelliCode.APIUsageExamples占用了。先把Edit.IntelliCode.A
安装数据库首次应用 Array_06 java oracle sql
可是为什么再一次失败之后就变成直接跳过那个要求 enter full pathname of java.exe的界面这个java.exe是你的Oracle 11g安装目录中例如：【F:\app\chen\product\11.2.0\dbhome_1\jdk\jre\bin】下的java.exe 。不是你的电脑安装的java jdk下的java.exe！注意第一次，使用SQL D
Weblogic Server Console密码修改和遗忘解决方法 bijian1013 Welogic
在工作中一同事将Weblogic的console的密码忘记了，通过网上查询资料解决，实践整理了一下。一.修改Console密码打开weblogic控制台，安全领域 --> myrealm -->&n
IllegalStateException: Cannot forward a response that is already committed Cwind java Servlets
对于初学者来说，一个常见的误解是：当调用 forward() 或者 sendRedirect() 时控制流将会自动跳出原函数。标题所示错误通常是基于此误解而引起的。示例代码： protected void doPost() { if (someCondition) { sendRedirect(); } forward(); // Thi
基于流的装饰设计模式木zi_鸣设计模式
当想要对已有类的对象进行功能增强时，可以定义一个类，将已有对象传入，基于已有的功能，并提供加强功能。自定义的类成为装饰类模仿BufferedReader，对Reader进行包装，体现装饰设计模式装饰类通常会通过构造方法接受被装饰的对象，并基于被装饰的对象功能，提供更强的功能。装饰模式比继承灵活，避免继承臃肿，降低了类与类之间的关系装饰类因为增强已有对象，具备的功能该
Linux中的uniq命令被触发 linux
Linux命令uniq的作用是过滤重复部分显示文件内容，这个命令读取输入文件，并比较相邻的行。在正常情况下，第二个及以后更多个重复行将被删去，行比较是根据所用字符集的排序序列进行的。该命令加工后的结果写到输出文件中。输入文件和输出文件必须不同。如果输入文件用“- ”表示，则从标准输入读取。 AD： uniq [选项] 文件说明：这个命令读取输入文件，并比较相邻的行。在正常情况下，第二个
正则表达式Pattern 肆无忌惮_ Pattern
正则表达式是符合一定规则的表达式，用来专门操作字符串，对字符创进行匹配，切割，替换，获取。例如，我们需要对QQ号码格式进行检验规则是长度6~12位不能0开头只能是数字，我们可以一位一位进行比较，利用parseLong进行判断，或者是用正则表达式来匹配[1-9][0-9]{4,14} 或者 [1-9]\d{4,14} &nbs
Oracle高级查询之OVER (PARTITION BY ..) 知了ing oracle sql
一、rank()/dense_rank() over(partition by ...order by ...) 现在客户有这样一个需求，查询每个部门工资最高的雇员的信息，相信有一定oracle应用知识的同学都能写出下面的SQL语句： select e.ename, e.job, e.sal, e.deptno from scott.emp e, (se
Python调试矮蛋蛋 python pdb
原文地址： http://blog.csdn.net/xuyuefei1988/article/details/19399137 1、下面网上收罗的资料初学者应该够用了，但对比IBM的Python 代码调试技巧： IBM：包括 pdb 模块、利用 PyDev 和 Eclipse 集成进行调试、PyCharm 以及 Debug 日志进行调试： http://www.ibm.com/d
webservice传递自定义对象时函数为空，以及boolean不对应的问题 alleni123 webservice
今天在客户端调用方法 NodeStatus status=iservice.getNodeStatus(). 结果NodeStatus的属性都是null。进行debug之后，发现服务器端返回的确实是有值的对象。后来发现原来是因为在客户端，NodeStatus的setter全部被我删除了。本来是因为逻辑上不需要在客户端使用setter，结果改了之后竟然不能获取带属性值的
java如何干掉指针，又如何巧妙的通过引用来操作指针————>说的就是java指针百合不是茶
C语言的强大在于可以直接操作指针的地址，通过改变指针的地址指向来达到更改地址的目的,又是由于c语言的指针过于强大，初学者很难掌握， java的出现解决了c，c++中指针的问题 java将指针封装在底层，开发人员是不能够去操作指针的地址，但是可以通过引用来间接的操作：定义一个指针p来指向a的地址（&是地址符号）：
Eclipse打不开，提示“An error has occurred.See the log file ***/.log” bijian1013 eclipse
打开eclipse工作目录的\.metadata\.log文件，发现如下错误： !ENTRY org.eclipse.osgi 4 0 2012-09-10 09:28:57.139 !MESSAGE Application error !STACK 1 java.lang.NoClassDefFoundError: org/eclipse/core/resources/IContai
spring aop实例annotation方法实现 bijian1013 java spring AOP annotation
在spring aop实例中我们通过配置xml文件来实现AOP，这里学习使用annotation来实现，使用annotation其实就是指明具体的aspect,pointcut和advice。1.申明一个切面(用一个类来实现)在这个切面里,包括了advice和pointcut AdviceMethods.jav
[Velocity一]Velocity语法基础入门 bit1129 velocity
用户和开发人员参考文档 http://velocity.apache.org/engine/releases/velocity-1.7/developer-guide.html 注释 1.行级注释## 2.多行注释#* *# 变量定义使用$开头的字符串是变量定义，例如$var1, $var2, 赋值使用#set为变量赋值，例
【Kafka十一】关于Kafka的副本管理 bit1129 kafka
1. 关于request.required.acks request.required.acks控制者Producer写请求的什么时候可以确认写成功，默认是0， 0表示即不进行确认即返回。 1表示Leader写成功即返回，此时还没有进行写数据同步到其它Follower Partition中 -1表示根据指定的最少Partition确认后才返回，这个在 Th
lua统计nginx内部变量数据 ronin47 lua nginx　统计
server { listen 80; server_name photo.domain.com; location /{set $str $uri; content_by_lua ' local url = ngx.var.uri local res = ngx.location.capture(
java-11.二叉树中节点的最大距离 bylijinnan java
import java.util.ArrayList; import java.util.List; public class MaxLenInBinTree { /* a. 1 / \ 2 3 / \ / \ 4 5 6 7 max=4 pass "root"
Netty源码学习-ReadTimeoutHandler bylijinnan java netty
ReadTimeoutHandler的实现思路：开启一个定时任务，如果在指定时间内没有接收到消息，则抛出ReadTimeoutException 这个异常的捕获，在开发中，交给跟在ReadTimeoutHandler后面的ChannelHandler，例如 private final ChannelHandler timeoutHandler = new ReadTim
jquery验证上传文件样式及大小(好用) cngolon 文件上传 jquery验证
<!DOCTYPE html> <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <script src="jquery1.8/jquery-1.8.0.
浏览器兼容【转】 cuishikuan css 浏览器 IE
浏览器兼容问题一：不同浏览器的标签默认的外补丁和内补丁不同问题症状：随便写几个标签，不加样式控制的情况下，各自的margin 和padding差异较大。碰到频率:100% 解决方案：CSS里 *{margin:0;padding:0;} 备注：这个是最常见的也是最易解决的一个浏览器兼容性问题，几乎所有的CSS文件开头都会用通配符*来设
Shell特殊变量：Shell $0, $#, $*, $@, $?, $$和命令行参数 daizj shell $#$?特殊变量
前面已经讲到，变量名只能包含数字、字母和下划线，因为某些包含其他字符的变量有特殊含义，这样的变量被称为特殊变量。例如，$ 表示当前Shell进程的ID，即pid，看下面的代码： $echo $$ 运行结果 29949 特殊变量列表变量含义 $0 当前脚本的文件名 $n 传递给脚本或函数的参数。n 是一个数字，表示第几个参数。例如，第一个
程序设计KISS 原则-------KEEP IT SIMPLE, STUPID! dcj3sjt126com unix
翻到一本书，讲到编程一般原则是kiss：Keep It Simple, Stupid.对这个原则深有体会，其实不仅编程如此，而且系统架构也是如此。 KEEP IT SIMPLE, STUPID! 编写只做一件事情，并且要做好的程序；编写可以在一起工作的程序，编写处理文本流的程序，因为这是通用的接口。这就是UNIX哲学.所有的哲学真正的浓缩为一个铁一样的定律，高明的工程师的神圣的“KISS 原
android Activity间List传值 dcj3sjt126com Activity
第一个Activity： import java.util.ArrayList;import java.util.HashMap;import java.util.List;import java.util.Map;import android.app.Activity;import android.content.Intent;import android.os.Bundle;import a
tomcat 设置java虚拟机内存 eksliang tomcat 内存设置
转载请出自出处：http://eksliang.iteye.com/blog/2117772 http://eksliang.iteye.com/ 常见的内存溢出有以下两种: java.lang.OutOfMemoryError: PermGen space java.lang.OutOfMemoryError: Java heap space ------------
Android 数据库事务处理 gqdy365 android
使用SQLiteDatabase的beginTransaction()方法可以开启一个事务，程序执行到endTransaction() 方法时会检查事务的标志是否为成功，如果程序执行到endTransaction()之前调用了setTransactionSuccessful() 方法设置事务的标志为成功则提交事务，如果没有调用setTransactionSuccessful() 方法则回滚事务。事
Java 打开浏览器 hw1287789687 打开网址 open浏览器 open browser 打开url 打开浏览器
使用java 语言如何打开浏览器呢? 我们先研究下在cmd窗口中,如何打开网址使用IE 打开 D:\software\bin>cmd /c start iexplore http://hw1287789687.iteye.com/blog/2153709 使用火狐打开 D:\software\bin>cmd /c start firefox http://hw1287789
ReplaceGoogleCDN：将 Google CDN 替换为国内的 Chrome 插件 justjavac chrome Google google api chrome插件
Chrome Web Store 安装地址： https://chrome.google.com/webstore/detail/replace-google-cdn/kpampjmfiopfpkkepbllemkibefkiice 由于众所周知的原因，只需替换一个域名就可以继续使用Google提供的前端公共库了。同样，通过script标记引用这些资源，让网站访问速度瞬间提速吧
进程VS.线程 m635674608 线程
资料来源： http://www.liaoxuefeng.com/wiki/001374738125095c955c1e6d8bb493182103fac9270762a000/001397567993007df355a3394da48f0bf14960f0c78753f000 1、Apache最早就是采用多进程模式 2、IIS服务器默认采用多线程模式 3、多进程优缺点优点：多进程模式最大
Linux下安装MemCached 字符串 memcached
前提准备：1. MemCached目前最新版本为：1.4.22，可以从官网下载到。2. MemCached依赖libevent，因此在安装MemCached之前需要先安装libevent。2.1 运行下面命令，查看系统是否已安装libevent。[root@SecurityCheck ~]# rpm -qa|grep libevent libevent-headers-1.4.13-4.el6.n
java设计模式之--jdk动态代理（实现aop编程） Supanccy2013 java DAO 设计模式 AOP
与静态代理类对照的是动态代理类，动态代理类的字节码在程序运行时由Java反射机制动态生成，无需程序员手工编写它的源代码。动态代理类不仅简化了编程工作，而且提高了软件系统的可扩展性，因为Java 反射机制可以生成任意类型的动态代理类。java.lang.reflect 包中的Proxy类和InvocationHandler 接口提供了生成动态代理类的能力。 &
Spring 4.2新特性-对java8默认方法(default method)定义Bean的支持 wiselyman spring 4
2.1 默认方法(default method) java8引入了一个default medthod; 用来扩展已有的接口,在对已有接口的使用不产生任何影响的情况下,添加扩展使用default关键字 Spring 4.2支持加载在默认方法里声明的bean 2.2 将要被声明成bean的类 public class DemoService {