nvidia-smi GPU卡信息获取

nvidia-smi GPU卡信息获取

1.GPU Initialization & Info

# nvidia-smi Tue Jan  9 14:39:59 2024        +---------------------------------------------------------------------------------------+ | NVIDIA-SMI 535.146.02             Driver Version: 535.146.02   CUDA Version: 12.2     | |-----------------------------------------+----------------------+----------------------+ | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC | | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. | |                                         |                      |               MIG M. | |=========================================+======================+======================| |   0  NVIDIA GeForce RTX 3060        Off | 00000000:01:00.0 Off |                  N/A | |  0%   53C    P8              18W / 170W |   2785MiB / 12288MiB |      0%      Default | |                                         |                      |                  N/A | +-----------------------------------------+----------------------+----------------------+                                                                                           +---------------------------------------------------------------------------------------+ | Processes:                                                                            | |  GPU   GI   CI        PID   Type   Process name                            GPU Memory | |        ID   ID                                                             Usage      | |=======================================================================================| |    0   N/A  N/A     40200      C   python3                                    2780MiB | +---------------------------------------------------------------------------------------+

2.GPU Status Query

# nvidia-smi -L GPU 0: NVIDIA GeForce RTX 3060 (UUID: GPU-543d8aa8-ce2f-5b19-a98a-6a1e7f111dee)

3.GPU Details

# nvidia-smi --query-gpu=index,name,uuid,serial --format=csv index, name, uuid, serial 0, NVIDIA GeForce RTX 3060, GPU-543d8aa8-ce2f-5b19-a98a-6a1e7f111dee, [N/A]

4.Monitor GPU Usage

# nvidia-smi dmon # gpu    pwr  gtemp  mtemp     sm    mem    enc    dec    jpg    ofa   mclk   pclk # Idx      W      C      C      %      %      %      %      %      %    MHz    MHz     0     19     56      -      0      0      0      0      0      0    405    210     0     19     56      -      0      0      0      0      0      0    405    210     0     19     56      -      0      0      0      0      0      0    405    210 ^C    0     19     56      -      0      0      0      0      0      0    405    210

5.Monitor GPU Processes

# nvidia-smi pmon # gpu         pid  type    sm    mem    enc    dec    command # Idx           #   C/G     %      %      %      %    name     0      40200     C      -      -      -      -    python3             0      40200     C      -      -      -      -    python3             0      40200     C      -      -      -      -    python3

6.List of Available Clocks

# nvidia-smi -q -d SUPPORTED_CLOCKS ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:44:31 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     Supported Clocks         Memory                            : 7501 MHz             Graphics                      : 2100 MHz             Graphics                      : 2092 MHz             Graphics                      : 2085 MHz             Graphics                      : 2077 MHz             Graphics                      : 2070 MHz             Graphics                      : 2062 MHz             Graphics                      : 2055 MHz             Graphics                      : 2047 MHz             Graphics                      : 2040 MHz             Graphics                      : 2032 MHz             Graphics                      : 2025 MHz             Graphics                      : 2017 MHz             Graphics                      : 2010 MHz .......

7.Current GPU Clock Speed

# nvidia-smi -q -d CLOCK ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:46:20 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     Clocks         Graphics                          : 210 MHz         SM                                : 210 MHz         Memory                            : 405 MHz         Video                             : 555 MHz     Applications Clocks         Graphics                          : N/A         Memory                            : N/A     Default Applications Clocks         Graphics                          : N/A         Memory                            : N/A     Deferred Clocks         Memory                            : N/A     Max Clocks         Graphics                          : 2100 MHz         SM                                : 2100 MHz         Memory                            : 7501 MHz         Video                             : 1950 MHz     Max Customer Boost Clocks         Graphics                          : N/A     SM Clock Samples         Duration                          : Not Found         Number of Samples                 : Not Found         Max                               : Not Found         Min                               : Not Found         Avg                               : Not Found     Memory Clock Samples         Duration                          : Not Found         Number of Samples                 : Not Found         Max                               : Not Found         Min                               : Not Found         Avg                               : Not Found     Clock Policy         Auto Boost                        : N/A         Auto Boost Default                : N/A

8.GPU Performance

# nvidia-smi -q -d PERFORMANCE ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:47:08 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     Performance State                     : P8     Clocks Event Reasons         Idle                              : Active         Applications Clocks Setting       : Not Active         SW Power Cap                      : Not Active         HW Slowdown                       : Not Active             HW Thermal Slowdown           : Not Active             HW Power Brake Slowdown       : Not Active         Sync Boost                        : Not Active         SW Thermal Slowdown               : Not Active         Display Clock Setting             : Not Active

9.GPU Topology

# nvidia-smi topo --matrix         GPU0    CPU Affinity    NUMA Affinity   GPU NUMA ID GPU0     X      0-7     0               N/A Legend:   X    = Self   SYS  = Connection traversing PCIe as well as the SMP interconnect between NUMA nodes (e.g., QPI/UPI)   NODE = Connection traversing PCIe as well as the interconnect between PCIe Host Bridges within a NUMA node   PHB  = Connection traversing PCIe as well as a PCIe Host Bridge (typically the CPU)   PXB  = Connection traversing multiple PCIe bridges (without traversing the PCIe Host Bridge)   PIX  = Connection traversing at most a single PCIe bridge   NV#  = Connection traversing a bonded set of # NVLinks

10.NVLink Status

# nvidia-smi nvlink --status GPU 0: NVIDIA A100 80GB PCIe (UUID: GPU-84ccface-663f-f5fd-8e8e-109d0f78bd2f) Link 0:

11.Display GPU Details

# nvidia-smi -i 0 -q ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:49:16 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     Product Name                          : NVIDIA GeForce RTX 3060     Product Brand                         : GeForce     Product Architecture                  : Ampere     Display Mode                          : Disabled     Display Active                        : Disabled     Persistence Mode                      : Disabled     Addressing Mode                       : None     MIG Mode         Current                           : N/A         Pending                           : N/A     Accounting Mode                       : Disabled ......

12.GPU App Details

# nvidia-smi -i 0 -q -d MEMORY,UTILIZATION,POWER,CLOCK,COMPUTE ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:50:08 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     FB Memory Usage         Total                             : 12288 MiB         Reserved                          : 243 MiB         Used                              : 2785 MiB         Free                              : 9259 MiB     BAR1 Memory Usage         Total                             : 256 MiB         Used                              : 4 MiB         Free                              : 252 MiB     Conf Compute Protected Memory Usage         Total                             : 0 MiB         Used                              : 0 MiB         Free                              : 0 MiB     Compute Mode                          : Default     Utilization         Gpu                               : 0 %         Memory                            : 0 %         Encoder                           : 0 %         Decoder                           : 0 %         JPEG                              : 0 %         OFA                               : 0 %     GPU Utilization Samples         Duration                          : 14.00 sec         Number of Samples                 : 71         Max                               : 0 %         Min                               : 0 %         Avg                               : 0 %     Memory Utilization Samples         Duration                          : 14.00 sec         Number of Samples                 : 71         Max                               : 0 %         Min                               : 0 %         Avg                               : 0 %     ENC Utilization Samples         Duration                          : 14.00 sec         Number of Samples                 : 71         Max                               : 0 %         Min                               : 0 %         Avg                               : 0 %

13.Change GPU Performance Level

# nvidia-smi -q -d PERFORMANCE ==============NVSMI LOG============== Timestamp                                 : Tue Jan  9 14:50:58 2024 Driver Version                            : 535.146.02 CUDA Version                              : 12.2 Attached GPUs                             : 1 GPU 00000000:01:00.0     Performance State                     : P8     Clocks Event Reasons         Idle                              : Active         Applications Clocks Setting       : Not Active         SW Power Cap                      : Not Active         HW Slowdown                       : Not Active             HW Thermal Slowdown           : Not Active             HW Power Brake Slowdown       : Not Active         Sync Boost                        : Not Active         SW Thermal Slowdown               : Not Active         Display Clock Setting             : Not Active