在3Dslicer中使用opencv连接相机

       前面已经介绍了创建扩展后的默认扩展程序，下面将修改这一扩展程序调用opencv库连接相机并在3Dslicer中显示出来。

首先点击扩展窗口中的Edit UI按钮，slicer会加载自带的Qt Designer工具。

        在Qt Designer中删除不需要的控件，如下图红色方框所示。

        此外还需要删除“弹簧”，去掉垂直间距。

 在Qt Designer中搜索label控件，放入右侧界面中，并双击删除上面的文字“TextLabel”。

  接下来，给窗体上的两个控件命名：

 此外，我们给按键重新命名，如下图更改text：

 

 至此，在Qt Designer中做的工作就完成了。

  然后单击Edit打开pycharm编辑器。

 默认程序中的测试类在此例程中未用到，可以直接删除；逻辑类在此例也未用到，可以做删除或隐藏。 

首先更改窗体类中的setup函数。将addObserver等不需要的程序删掉，只保留我们所需要的加载UI文件、设置mrml场景、设置控件槽函数（方便打开和关闭相机）功能等。修改后的代码如下：

  def setup(self):
    ScriptedLoadableModuleWidget.setup(self)

    uiWidget = slicer.util.loadUI(self.resourcePath('UI/Camera.ui'))
    self.layout.addWidget(uiWidget)
    self.ui = slicer.util.childWidgetVariables(uiWidget)

    uiWidget.setMRMLScene(slicer.mrmlScene)

    self.logic = CameraLogic()
    # Buttons
    self.ui.applyButton.connect('clicked(bool)', self.onApplyButton)
    self.initializeParameterNode()
    self.ui.applyButton.enabled = True

 然后在其他九个功能函数中删除我们不需要的功能（几乎全删）。前面提到，这些功能函数主要做了参数加载更新等操作，如果未全部删除重新加载的时候会报错。删完后的代码如下所示：

  def cleanup(self):
    pass
  def enter(self):
    self.initializeParameterNode()
  def exit(self):
    pass
  def onSceneStartClose(self, caller, event):
    self.setParameterNode(None)
  def onSceneEndClose(self, caller, event):
    if self.parent.isEntered:
      self.initializeParameterNode()
  def initializeParameterNode(self):
    self.setParameterNode()
  def setParameterNode(self):
    self.updateGUIFromParameterNode()
  def updateGUIFromParameterNode(self, caller=None, event=None):
    pass
  def updateParameterNodeFromGUI(self, caller=None, event=None):
    if self._parameterNode is None or self._updatingGUIFromParameterNode:
      return
    wasModified = self._parameterNode.StartModify()  # Modify all properties in a single batch
    self._parameterNode.EndModify(wasModified)

此外我们需要在窗体类的初始化函数中初始化qt定时器和它的中断时间。

class CameraWidget(ScriptedLoadableModuleWidget, VTKObservationMixin):
  def __init__(self, parent=None):
    ScriptedLoadableModuleWidget.__init__(self, parent)
    VTKObservationMixin.__init__(self)  # needed for parameter node observation
    self.logic = None
    self._parameterNode = None
    self._updatingGUIFromParameterNode = False
    self.ui = None
    self.timer = qt.QTimer()
    self.timer.setInterval(20)
    print("界面加载成功")

在按键槽函数中加入如下图所示代码，里面完成了三件事：①按键状态更新②定时器开启和中断③相机视频开启、设置和中断。定时器连接了一个槽函数，这是我们在其中创建一个新函数，用于读取视频数据并放入Label控件中显示。

 def onApplyButton(self):
    with slicer.util.tryWithErrorDisplay("Failed to compute results.", waitCursor=True):

      # Compute output
      if self.ui.applyButton.text =="Start":
        self.ui.applyButton.text = "Stop"
        self.timer.connect('timeout()', self.ImageShow)
        self.timer.start()
        self.cap = cv2.VideoCapture(0)
        print("相机打开成功!")
        self.cap.set(cv2.CAP_PROP_BUFFERSIZE, 3)
        self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1024)
        self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 768)
      else:
        self.ui.applyButton.text = "Start"
        self.timer.stop()
        self.cap.release()
        print("关闭相机成功！")

上述定时器槽函数代码如下所示（新增加的函数）：

 def ImageShow(self):
    try:
      if self.cap.isOpened() != 1:
        print("相机打开失败！")
        return
      print("相机运行中")
      ret, img = self.cap.read()
      showImg = cv2.resize(img,(640,480))

      # QtImg = qt.QImage(showImg.data,qt.QImage.Format_RGB888)
      # # QtImg = qt.QImage(showImg.data,showImg.shape[1],showImg.shape[0],showImg.shape[2]*showImg.shape[1],qt.QImage.Format_RGB888)
      # self.ui.ImageShowLabel.clear()
      # self.ui.ImageShowLabel.setPixmap(qt.QPixmap.fromImage(QtImg))

      # 测试有效
      cv2.imwrite("./Work/Camera/Camera/Resources/grab.jpg", showImg, [cv2.IMWRITE_PNG_BILEVEL, 0])
      QtImg1 = qt.QPixmap("./Work/Camera/Camera/Resources/grab.jpg") # 从文件夹中获取图像有效
      self.ui.ImageShowLabel.setPixmap(QtImg1)

      # self.ui.ImageShowLabel.setPixmap(qt.QPixmap.fromImage(QtImg1))
    except Exception as e:
      print('%s' % e)

      值得注意的是这里使用的显示视频的方法，在代码中注释调的一部分是pyqt一般的label组件显示视频流的方法，但是在这里试验起来不成功(后续会尝试其他方法），所以改成了注释下面的程序内容，即先将获取到的图像数据以png格式存储起来，在使用QPixmap获取图像数据，最后放入label组件中显示。

测试效果如下所示：

 最后奉上完整程序。

import logging
import os
import ctk
import numpy as np
import vtk
import qt
import slicer
from slicer.ScriptedLoadableModule import *
from slicer.util import VTKObservationMixin

import cv2
#
# Camera
#

class Camera(ScriptedLoadableModule):
  """Uses ScriptedLoadableModule base class, available at:
  https://github.com/Slicer/Slicer/blob/master/Base/Python/slicer/ScriptedLoadableModule.py
  """

  def __init__(self, parent):
    ScriptedLoadableModule.__init__(self, parent)
    self.parent.title = "Camera"  # TODO: make this more human readable by adding spaces
    self.parent.categories = ["Examples"]  # TODO: set categories (folders where the module shows up in the module selector)
    self.parent.dependencies = []  # TODO: add here list of module names that this module requires
    self.parent.contributors = ["John Doe (AnyWare Corp.)"]  # TODO: replace with "Firstname Lastname (Organization)"
    # TODO: update with short description of the module and a link to online module documentation
    self.parent.helpText = """
This is an example of scripted loadable module bundled in an extension.
See more information in module documentation.
"""
    # TODO: replace with organization, grant and thanks
    self.parent.acknowledgementText = """
This file was originally developed by Jean-Christophe Fillion-Robin, Kitware Inc., Andras Lasso, PerkLab,
and Steve Pieper, Isomics, Inc. and was partially funded by NIH grant 3P41RR013218-12S1.
"""

    # Additional initialization step after application startup is complete
    slicer.app.connect("startupCompleted()", registerSampleData)


#
# Register sample data sets in Sample Data module
#

def registerSampleData():
  """
  Add data sets to Sample Data module.
  """
  # It is always recommended to provide sample data for users to make it easy to try the module,
  # but if no sample data is available then this method (and associated startupCompeted signal connection) can be removed.

  import SampleData
  iconsPath = os.path.join(os.path.dirname(__file__), 'Resources/Icons')

  # To ensure that the source code repository remains small (can be downloaded and installed quickly)
  # it is recommended to store data sets that are larger than a few MB in a Github release.

  # Camera1
  SampleData.SampleDataLogic.registerCustomSampleDataSource(
    # Category and sample name displayed in Sample Data module
    category='Camera1',
    sampleName='Camera1',
    # Thumbnail should have size of approximately 260x280 pixels and stored in Resources/Icons folder.
    # It can be created by Screen Capture module, "Capture all views" option enabled, "Number of images" set to "Single".
    thumbnailFileName=os.path.join(iconsPath, 'Camera1.png'),
    # Download URL and target file name
    uris="https://github.com/Slicer/SlicerTestingData/releases/download/SHA256/998cb522173839c78657f4bc0ea907cea09fd04e44601f17c82ea27927937b95",
    fileNames='Camera1.nrrd',
    # Checksum to ensure file integrity. Can be computed by this command:
    #  import hashlib; print(hashlib.sha256(open(filename, "rb").read()).hexdigest())
    checksums = 'SHA256:998cb522173839c78657f4bc0ea907cea09fd04e44601f17c82ea27927937b95',
    # This node name will be used when the data set is loaded
    nodeNames='Camera1'
  )

  # Camera2
  SampleData.SampleDataLogic.registerCustomSampleDataSource(
    # Category and sample name displayed in Sample Data module
    category='Camera2',
    sampleName='Camera2',
    thumbnailFileName=os.path.join(iconsPath, 'Camera2.png'),
    # Download URL and target file name
    uris="https://github.com/Slicer/SlicerTestingData/releases/download/SHA256/1a64f3f422eb3d1c9b093d1a18da354b13bcf307907c66317e2463ee530b7a97",
    fileNames='Camera2.nrrd',
    checksums = 'SHA256:1a64f3f422eb3d1c9b093d1a18da354b13bcf307907c66317e2463ee530b7a97',
    # This node name will be used when the data set is loaded
    nodeNames='Camera2'
  )


#
# CameraWidget
#

class CameraWidget(ScriptedLoadableModuleWidget, VTKObservationMixin):
  def __init__(self, parent=None):
    """
    Called when the user opens the module the first time and the widget is initialized.
    """
    ScriptedLoadableModuleWidget.__init__(self, parent)
    VTKObservationMixin.__init__(self)  # needed for parameter node observation
    self.logic = None
    self._parameterNode = None
    self._updatingGUIFromParameterNode = False
    self.ui = None
    self.timer = qt.QTimer()
    self.timer.setInterval(20)
    print("界面加载成功")

  def setup(self):
    ScriptedLoadableModuleWidget.setup(self)

    uiWidget = slicer.util.loadUI(self.resourcePath('UI/Camera.ui'))
    self.layout.addWidget(uiWidget)
    self.ui = slicer.util.childWidgetVariables(uiWidget)

    uiWidget.setMRMLScene(slicer.mrmlScene)

    self.logic = CameraLogic()
    # Buttons
    self.ui.applyButton.connect('clicked(bool)', self.onApplyButton)
    self.initializeParameterNode()
    self.ui.applyButton.enabled = True

  def cleanup(self):
    pass
  def enter(self):
    self.initializeParameterNode()
  def exit(self):
    pass
  def onSceneStartClose(self, caller, event):
    self.setParameterNode(None)
  def onSceneEndClose(self, caller, event):
    if self.parent.isEntered:
      self.initializeParameterNode()
  def initializeParameterNode(self):
    self.setParameterNode()
  def setParameterNode(self):
    self.updateGUIFromParameterNode()
  def updateGUIFromParameterNode(self, caller=None, event=None):
    pass
  def updateParameterNodeFromGUI(self, caller=None, event=None):
    if self._parameterNode is None or self._updatingGUIFromParameterNode:
      return
    wasModified = self._parameterNode.StartModify()  # Modify all properties in a single batch
    self._parameterNode.EndModify(wasModified)

  def onApplyButton(self):
    with slicer.util.tryWithErrorDisplay("Failed to compute results.", waitCursor=True):

      # Compute output
      if self.ui.applyButton.text =="Start":
        self.ui.applyButton.text = "Stop"
        self.timer.connect('timeout()', self.ImageShow)
        self.timer.start()
        self.cap = cv2.VideoCapture(0)
        print("相机打开成功!")
        self.cap.set(cv2.CAP_PROP_BUFFERSIZE, 3)
        self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, 1024)
        self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 768)
      else:
        self.ui.applyButton.text = "Start"
        self.timer.stop()
        self.cap.release()
        print("关闭相机成功！")

  def ImageShow(self):
    try:
      if self.cap.isOpened() != 1:
        print("相机打开失败！")
        return
      print("相机运行中")
      ret, img = self.cap.read()
      showImg = cv2.resize(img,(640,480))

      # QtImg = qt.QImage(showImg.data,qt.QImage.Format_RGB888)
      # # QtImg = qt.QImage(showImg.data,showImg.shape[1],showImg.shape[0],showImg.shape[2]*showImg.shape[1],qt.QImage.Format_RGB888)
      # self.ui.ImageShowLabel.clear()
      # self.ui.ImageShowLabel.setPixmap(qt.QPixmap.fromImage(QtImg))

      # 测试有效
      cv2.imwrite("./Work/Camera/Camera/Resources/grab.jpg", showImg, [cv2.IMWRITE_PNG_BILEVEL, 0])
      QtImg1 = qt.QPixmap("./Work/Camera/Camera/Resources/grab.jpg") # 从文件夹中获取图像有效
      self.ui.ImageShowLabel.setPixmap(QtImg1)

      # self.ui.ImageShowLabel.setPixmap(qt.QPixmap.fromImage(QtImg1))
    except Exception as e:
      print('%s' % e)

#
# CameraLogic
#

class CameraLogic(ScriptedLoadableModuleLogic):
  def __init__(self):
    ScriptedLoadableModuleLogic.__init__(self)
  def setDefaultParameters(self):
    print("no DefaultParameters")
  def process(self):
    print("non")