Message Board

Dear readers, authors and reviewers,you can add a message on this page. We will reply to you as soon as possible!

2021 Volume 43 Issue 10
Article Contents

ZHANG Fan, YANG Yong, LUO Shaoming, et al. Research of Online Vision Detection for Germination of Plug Seedlings[J]. Journal of Southwest University Natural Science Edition, 2021, 43(10): 84-91. doi: 10.13718/j.cnki.xdzk.2021.10.011
Citation: ZHANG Fan, YANG Yong, LUO Shaoming, et al. Research of Online Vision Detection for Germination of Plug Seedlings[J]. Journal of Southwest University Natural Science Edition, 2021, 43(10): 84-91. doi: 10.13718/j.cnki.xdzk.2021.10.011

Research of Online Vision Detection for Germination of Plug Seedlings

More Information
  • Corresponding author: YANG Yong ; 
  • Received Date: 08/06/2020
    Available Online: 20/10/2021
  • MSC: S62

  • China is a large vegetable production and consumption country, and the demand for vegetable planting is very huge. At present, more than 60% of vegetable planting adopts seedling transplanting, and the traditional transplanting operation requires a large number of manual operations, which restricts the development of vegetable cultivation. Facility cultivation has developed rapidly in recent years due to its little dependence on weather, safety and pollutionlessness. Although reseeding can improve the efficiency of plug seedling transplanting, it wastes seeds and manpower and restricts the improvement of the whole production of plant facto ries. Large-scale, high-speed and automatic operation requires fast and accurate sensing of the germination of plug seedlings and the growth information of transplanted seedlings, and object detection is the key. Object detection not only recognizes object categories, but also predicts the location of each object. With the development of computer vision technology, the efficiency of object detection is greatly improved as a new solution which is provided to replace the traditional method of manual visual detection. In this paper, an online vision detection system based on machine learning is proposed, which is used to detect the germination rate of plug seedlings and calculate the growth direction, so as to localize the foothold of seedlings. This online vision detection system uses digital camera networking technology to collect the images of seed germination, makes training based on the machine learning method, constructs atraining set and atest set, calculates the dependability of each foothold of seedling by multi-view fusion algorithm, and judges the germination situation according to the dependability. The results of an experiment show that the single view has fast detection speed and low false detection rate, but low accuracy, whilethe adoption of multi-view fusiongives asuccess rate of plug-seedling detection of 95.16% and asuccess rate of seedling location of 100%. In conclusion, the on-line multi-view fusion system proposed in this paper has relatively high detection accuracy, though there is room for further improvement, and may provide technical references for mass, high-speed and automatic production requirements of plant facto ries.
  • 加载中
  • [1] SHAMSHIRI R, KALANTARI F, TING K C, et al. Advances in Greenhouse Automation and Controlled Environment Agriculture: A Transition to Plant Factories and Urban Agriculture[J]. International Journal of Agricultural and Biological Engineering, 2018, 11(1): 1-22.

    Google Scholar

    [2] 刘霓红, 蒋先平, 程俊峰, 等. 国外有机设施园艺现状及对中国设施农业可持续发展的启示[J]. 农业工程学报, 2018, 34(15): 1-9. doi: 10.11975/j.issn.1002-6819.2018.15.001

    CrossRef Google Scholar

    [3] 童俊华, 俞高红, 朱赢鹏, 等. 三臂回转式蔬菜钵苗取苗机构设计与试验[J]. 农业机械学报, 2019, 50(1): 113-121.

    Google Scholar

    [4] 齐飞, 李恺, 李邵, 等. 世界设施园艺智能化装备发展对中国的启示研究[J]. 农业工程学报, 2019, 35(2): 183-195.

    Google Scholar

    [5] 束胜, 康云艳, 王玉, 等. 世界设施园艺发展概况、特点及趋势分析[J]. 中国蔬菜, 2018(7): 1-13.

    Google Scholar

    [6] MUANGPRATHUB J, BOONNAM N, KAJORNKASIRAT S, et al. IoT and Agriculture Data Analysis for Smart farm[J]. Computers and Electronics in Agriculture, 2019, 156: 467-474. doi: 10.1016/j.compag.2018.12.011

    CrossRef Google Scholar

    [7] 齐飞, 魏晓明, 张跃峰. 中国设施园艺装备技术发展现状与未来研究方向[J]. 农业工程学报, 2017, 33(24): 1-9. doi: 10.11975/j.issn.1002-6819.2017.24.001

    CrossRef Google Scholar

    [8] 李玉东, 于晓峰, 佟立辉. 玉米单粒播种栽培技术[J]. 农业与技术, 2013, 33(11): 139. doi: 10.3969/j.issn.1671-962X.2013.11.115

    CrossRef Google Scholar

    [9] 胡建平, 常航, 杨丽红, 等. 自动移栽机整排取苗间隔投苗控制系统设计与试验[J]. 农业机械学报, 2018, 49(6): 78-84.

    Google Scholar

    [10] 王桂莲, 刘伟超, 王安, 等. 基于机器视觉的水稻秧盘育秧智能补种装置设计与试验[J]. 农业工程学报, 2018, 34(13): 35-42. doi: 10.11975/j.issn.1002-6819.2018.13.005

    CrossRef Google Scholar

    [11] 玄冠涛, 邵园园, 侯加林, 等. 小型蔬菜穴盘精密播种机的设计与试验[J]. 农机化研究, 2018, 40(2): 85-89.

    Google Scholar

    [12] 朱盘安, 李建平, 楼建忠, 等. 便携式蔬菜穴盘自动播种机设计与试验[J]. 农业机械学报, 2016, 47(8): 7-13.

    Google Scholar

    [13] 廖庆喜, 雷小龙, 廖宜涛, 等. 油菜精量播种技术研究进展[J]. 农业机械学报, 2017, 48(9): 1-16.

    Google Scholar

    [14] REHMAN T U, MAHMUD M S, CHANG Y K, et al. Current and Future Applications of Statistical Machine Learning Algorithms for Agricultural Machine Vision Systems[J]. Computers and Electronics in Agriculture, 2019, 156: 585-605. doi: 10.1016/j.compag.2018.12.006

    CrossRef Google Scholar

    [15] 全胜, 基于机器视觉的蔬菜种子质量检测系统的设计与实现[D]. 长沙: 湖南大学, 2017.

    Google Scholar

    [16] 豆东东. 基于机器视觉的大棚蔬菜生长状况的监测[D]. 上海: 东华大学, 2016.

    Google Scholar

    [17] 冯青春, 陈建, 李翠玲, 等. 基于光度立体视觉的蔬菜秧苗叶片形态测量方法[J]. 农业机械学报, 2018, 49(5): 43-50.

    Google Scholar

    [18] 冯甲一. 基于计算机视觉技术的叶类蔬菜新鲜度检测分级研究[D]. 南京: 南京农业大学, 2012.

    Google Scholar

    [19] 张雨滋. 计算机视觉技术在果园除草机上的应用研究[J]. 农机化研究, 2018, 40(3): 208-211, 216. doi: 10.3969/j.issn.1003-188X.2018.03.042

    CrossRef Google Scholar

    [20] 潘琪, 尹雄, 秦襄培, 等. 基于HOG特征与SVM的胶体气泡识别方法研究[J]. 智能计算机与应用, 2018, 8(5): 21-24. doi: 10.3969/j.issn.2095-2163.2018.05.005

    CrossRef Google Scholar

    [21] 郭晶晶, 许萌, 孔令爱. 基于改进HOG特征的人数统计算法[J]. 信息技术与信息化, 2019(3): 81-84.

    Google Scholar

    [22] 王伟, 吴芳. 基于注意机制和循环卷积神经网络的细粒度图像分类算法[J]. 西南师范大学学报(自然科学版), 2020, 45(1): 48-56.

    Google Scholar

    [23] KING D E. Dlib-ml: A Machine Learning Toolkit[J]. Journal of Machine Learning Research, 2009, 10(3): 1755-1758.

    Google Scholar

    [24] DALAL N, TRIGGS B. Histograms of Oriented Gradients for Human Detection[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), June 20-25, 2005, San Diego, CA, USA, 2005: 886-893.

    Google Scholar

    [25] CHANG C C, LIN C J. LIBSVM: a Library for Support Vector Machines[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2011, 2 (3): 27.

    Google Scholar

  • 加载中
通讯作者: 陈斌, bchen63@163.com
  • 1. 

    沈阳化工大学材料科学与工程学院 沈阳 110142

  1. 本站搜索
  2. 百度学术搜索
  3. 万方数据库搜索
  4. CNKI搜索

Figures(5)  /  Tables(1)

Article Metrics

Article views(2950) PDF downloads(202) Cited by(0)

Access History

Other Articles By Authors

Research of Online Vision Detection for Germination of Plug Seedlings

    Corresponding author: YANG Yong ; 

Abstract: China is a large vegetable production and consumption country, and the demand for vegetable planting is very huge. At present, more than 60% of vegetable planting adopts seedling transplanting, and the traditional transplanting operation requires a large number of manual operations, which restricts the development of vegetable cultivation. Facility cultivation has developed rapidly in recent years due to its little dependence on weather, safety and pollutionlessness. Although reseeding can improve the efficiency of plug seedling transplanting, it wastes seeds and manpower and restricts the improvement of the whole production of plant facto ries. Large-scale, high-speed and automatic operation requires fast and accurate sensing of the germination of plug seedlings and the growth information of transplanted seedlings, and object detection is the key. Object detection not only recognizes object categories, but also predicts the location of each object. With the development of computer vision technology, the efficiency of object detection is greatly improved as a new solution which is provided to replace the traditional method of manual visual detection. In this paper, an online vision detection system based on machine learning is proposed, which is used to detect the germination rate of plug seedlings and calculate the growth direction, so as to localize the foothold of seedlings. This online vision detection system uses digital camera networking technology to collect the images of seed germination, makes training based on the machine learning method, constructs atraining set and atest set, calculates the dependability of each foothold of seedling by multi-view fusion algorithm, and judges the germination situation according to the dependability. The results of an experiment show that the single view has fast detection speed and low false detection rate, but low accuracy, whilethe adoption of multi-view fusiongives asuccess rate of plug-seedling detection of 95.16% and asuccess rate of seedling location of 100%. In conclusion, the on-line multi-view fusion system proposed in this paper has relatively high detection accuracy, though there is room for further improvement, and may provide technical references for mass, high-speed and automatic production requirements of plant facto ries.

  • 开放科学(资源服务)标志码(OSID):

  • 植物工厂在未来城市农业现代化建设中发挥着重要作用. 我国是蔬菜栽培的农业生产大国,但不是农业生产强国,确保新鲜水果和蔬菜的安全高效供给面临巨大的挑战[1-2]. 目前,我国蔬菜的栽植量巨大,60%以上蔬菜种植采用育苗移栽,传统的移栽作业方式要求大量的手工作业,制约了蔬菜栽培的发展[3]. 设施园艺智能化是当前包括中国在内的世界农业智能化装备的研发热点和产业升级重点[4-6],自动化作业可以解放人的手工劳动,在保持质量稳定的同时使作业速度成倍提高,但目前距离产业化应用仍存在目标检测、种苗识别等技术瓶颈需要突破[7].

    发芽率是采购种子时计算成本的重要依据,对研究农作物适宜的环境条件亦有意义. 由于穴盘播种普遍存在空穴问题,为保证出苗率,一般在1个穴位里放多粒种子,甚至通过补种获得整齐一致的穴盘苗[8]. 补种虽然可以提高穴盘的使用效率,但浪费种子,效率低,只适合人工操作,影响发芽率测算,也制约了移栽效率[9-10]. 随着计算机视觉和图像处理技术的进步,实施单粒精量播种有利于在线检测种子的发芽率和提高移栽效率[11-12]. 现代检测技术已经形成了自动化操作,逐步替代了人工目测的传统检测方法,产品的检测效率极大提高[13-14]. 目前,机器视觉已应用到农业生产的多个检测环节,如:种子的筛选和质量检验[15]、蔬菜生长状况的监测[16]、蔬菜秧苗形态测量[17]、蔬菜新鲜度分级[18]、杂草的检测[19]等. 配置视觉的检测系统可在线监测穴盘苗的发芽生长情况,不受空穴问题影响.

    本文设计的基于机器学习的发芽率在线视觉检测系统,是基于机器学习的图像特征提取方法和分类方法[20-22],实现对穴盘苗发芽阶段的图像检测[23-25];基于多视图定位原理,提出融合不同视角的检测结果计算穴孔的可信度;最后对系统进行试验,验证了穴盘苗检出率、穴位检出率和误检率等详细指标.

1.   系统简介
  • 穴盘苗发芽率在线视觉检测系统如图 1所示,主要由传送控制系统、支撑架、移栽机构、夹持手爪、检测相机、穴盘和传送带组成. 在育苗室,穴盘苗通过上料机推送至传送带后随传动带运动,检测相机对穴盘苗进行拍照,采集图像并输送至计算节点. 计算节点对发芽种苗进行图像识别,计算出种子的发芽情况和生长姿态以及所在的穴孔并保存记录.

  • 植物工厂是设施园艺的高级形式,由于种子出苗率的问题,在育苗阶段要检测发芽率、生长情况和种苗的位置信息,一般设施园艺生产中使用“单机检测”或“人工检测”,这种方式存在性能瓶颈,制约了穴盘的输送速度,无法满足高速的需求. 从规模效率的角度出发,在线视觉检测具有高速、非接触式的优点,满足植物工厂大规模、高速、高精度检测的需求.

    穴盘苗发芽率在线视觉检测系统工作原理是通过将多个视图检测结果融合达到提高检测精度,同时估算种苗所在穴位的效果,主要包括多相机采集图像、网络传输、机器学习方法快速检测和多视图检测结果融合算法等环节.

2.   穴盘苗视觉检测
  • 从油麦菜种子的穴盘苗图像可观测到发芽的情况,如图 2所示. 图片来自于顶部视角拍摄. 穴盘是黑色塑料制品,共有72个穴孔,采取6×12排列,每个穴孔放有1棵种子在椰糠混合土栽培基质中.

    客观存在各个穴孔发芽情况不一致及空穴的现象. 种子发芽除了本身发育完全的内在条件外,还受到外部环境条件的影响,外部环境条件包括水分、氧气、温度、光线等,对应的种苗发芽的不同阶段在图像上也呈现不同的效果,过早可能叶片还没有展开,过晚有的种苗已经长出多片叶子. 种苗发芽的方向和姿态、发芽的进度是系统检测发芽率的重要影响因素.

  • 种苗在颜色和形状上与背景存在明显的差异,用颜色特征可以区分种苗与背景,作为检测种苗的依据,这种用颜色特征的方法具有较高的鲁棒性. 通常相机采集的彩色图像为RGB模型,不能直接使用,要转换为在植物叶片检测中广泛使用的HSV模型. 本文先将获得的彩色图像进行颜色模型转换,然后将提取种苗所拥有的绿色并过滤掉绿色以外的颜色.

  • 对人而言,最直观的颜色模型是HSV(Hue色相、Saturation饱和度、Value/Brightness亮度,也叫HSB)模型,HSV模型是由A. R. Smith在1978年创建的一种颜色空间,也称六角锥体模型. 在自然界单色光有自己独特的光谱特征,HSV模型中H表示色彩信息,即光谱颜色所在的位置,提取种苗的绿色要选择对应的H值;饱和度S是一个比例值,表示所选颜色的纯度和该颜色最大的纯度之间的比率;V表示色彩的明亮程度,与光强度无直接关系. 用HS表示颜色距离,对于不同的彩色区域,混合HS变量,根据划定的阈值就可以进行简单快速的颜色分割. 在HSV色域中,色调H用角度表示,取值范围为H∈[0°,360°],从红色开始按逆时针方向计算,红色为0°,绿色为120°,蓝色为240°;饱和度S取值范围为S∈[0.0,1.0];亮度V取值范围为V∈[0.0,1.0].

    彩色图像从RGB模型转换为HSV模型,计算方法如下:

    首先,将图像的RGB值做归一化处理,即将每个通道的取值范围由(0~255)映射到(0~1.0),新的RGB模型计算,如公式(1)所示.

    取新的RGB模型的最大值Cmax和最小值Cmin,如公式(2)所示:

    求最大值与最小值的差,如公式(3)所示.

    公式(4)~(6)分别计算HSV的值.

  • 种苗的颜色是绿色,与穴盘、基质以及其他设备的颜色有明显的区别,因此利用HSV模型对绿色的连续表示,将绿色从背景颜色中提取出来,提取后的图像只包含绿色和去除背景部分的黑色,将绿色替换成白色就可以获得灰度图,效果如图 3所示.

  • 由于种苗发芽生长的姿态不同,机器要像人一样快速准确检测种苗是一项具有挑战性的工作. 在基于特征的目标检测研究上,局部归一化的方向梯度直方图(Histograms of Oriented Gradient,HOG)相比于传统的小波、SIFT、形状上下文方法有更好的表现[24]. HOG特征检测种苗的基本原理是利用图像的梯度和边缘方向的密度分布描述图像中种苗的外观和形状,HOG特征检测种苗的方法是提取纹理特征,不受颜色变化影响,对光照变化、阴影等噪声具有较好的鲁棒性. HOG只能用于彩色图像或灰度图,无法用于二值图.

    提取HOG特征后就需要进行分类,使用核方法的SVM是目前广泛使用的分类器,在图像分类任务上有很好的性能表现. SVM是在统计学习理论的基础上发展起来的机器学习算法,以结构风险最小化原理为准则,能够有效解决过拟合、落入局部极值和“维数灾难”等问题,在解决小样本、非线性、高维空间模式识别问题中表现出良好的推广性和较好的分类准确性.

  • 发芽种苗所在的穴位是一项重要的信息,可根据多视图检测结果分析得出. 之所以采用多视图,一方面是视觉检测存在漏检和误检;另一方面则是由于种苗生长方向的不确定性以及拍照角度的问题,检测结果误判种苗落在其他穴孔上.

    标准统一的工业产品使用单个摄像头即可完成准确的检测,但应用在种苗的穴位判断上,单个相机的检测结果不足以获得好的漏检率和误判率,如图 3所示. 通过顶视图观察,种苗的叶片在边界上,根系则可能是在上部分穴位,也可能是在下部分穴位,即使图像检测成功,仍然不能判断穴位;LT(左前)视图和RT(右前)视图观察种苗叶片在下部分穴位,根据图像检测结果判断穴位是误判;LB(左后)视图和RB(右后)视图观察种苗叶片在上部分穴位,根据图像检测结果判断穴位是正确的. 利用多个相机从不同视角采集图像信息进行融合后判断,不仅减少漏检,还可减小误判几率.

    多视图融合判断穴位算法描述如下:

    if detctor(TOP) at border:

      ret1=0.6

      ret2=check_location (LT | RT | LB | RB)

      ret3=check_ neighbors (TOP)

      location_dependability=ret1+ ret2+ ret3

    else if detctor(TOP) at center:

      ret1=high_location_dependability(TOP)

      ret2=high_location_dependability(LT | RT | LB | RB)

      location_dependability=ret1+ ret2

    else:

      if detctor (LT | RT | LB | RB) at border:

        location_dependability=low_location_dependability(LT | RT | LB | RB)

      else ifdetctor(LT | RT | LB | RB) at center:

        location_dependability=high_location_dependability(LT | RT | LB | RB)

      else:

        NULL,NO Seedling

3.   试验与结果分析
  • 为测试多视图融合下的发芽检测效果,本文采用小样本训练算法,然后分别对采集的5个视图图像进行检测,如图 4所示. 试验的穴盘共有72个穴位,其中左下方的1个穴位空着作为标记,有9个穴位的种子没有发芽,因此最终有62个穴位的种子成功发芽. 5个方向的检测试验得到的成功率各不相同,LT视图检测到42个,RT视图检测到44个,TOP视图检测到40个,LB视图检测到39个,RB视图检测到33个,分别对应图 4的LT,RT,TOP,LB和RB子图.

  • 穴盘一共播种71颗,发芽62颗,种子的实际发芽率为87.32%,其中各视图的检测发芽率和漏检数各有不同. LT视图的检测发芽率为$\frac{{{\rm{42}}}}{{{\rm{71}}}}$=59.15%,漏检测20个,穴盘苗检出率为67.74%;RT视图的检测发芽率为$\frac{{{\rm{44}}}}{{{\rm{71}}}}$=61.97%,漏检测18个,穴盘苗检出率为70.97%;TOP视图的检测发芽率为$\frac{{{\rm{40}}}}{{{\rm{71}}}}$=56.34%,漏检测22个,穴盘苗检出率为64.52%;LB视图的检测发芽率为$\frac{{{\rm{39}}}}{{{\rm{71}}}}$=54.93%,漏检测23个,穴盘苗检出率为62.90%;RT视图的检测发芽率为$\frac{{{\rm{33}}}}{{{\rm{71}}}}$=46.48%,漏检测29个,穴盘苗检出率为53.23%.

    由试验结果可以得到,单个相机的检测发芽率与实际发芽率存在较大偏差,存在漏检问题,所以穴盘苗的检出率低. 单个视图检测算法优化只能减小漏检,但不能消除漏检,而且减少漏检的同时会导致误检增多. 本文探讨多视图信息融合的方法提升检测效果,在无误检的前提下减少漏检,因此进一步做了穴盘苗定位试验.

  • 多视图检测结果画矩形框,框的中心分别用不同的符号表示. 符号“*”表示TOP视图检测结果框的中心,符号“o”表示LT和RT视图检测结果框的中心,符号“+”表示LB和RB视图检测结果框的中心,配上穴盘苗原图作为对比,检测结果叠加效果如图 5所示. 种子没有发芽的9个穴位分别是K1,I4,I5,G4,F3,E4,E6,C5和A4.

  • 图 5可观察到,多视图检测结果叠加后明显提升了检测效果,这里用可信度来描述. 采用多视图融合判断穴位算法对叠加效果进行计算,可得到每个穴位有种苗的可信度,如表 1所示. 除种子没有发芽的9个穴位之外,有3个穴位[I1,G3,A2] 漏检. 多视图融合后的检测发芽率为$\frac{{{\rm{59}}}}{{{\rm{71}}}}$=83.09%,漏检3个,穴盘苗检出率为$\frac{{{\rm{33}}}}{{{\rm{62}}}}$=95.16%. 有可信度的穴位实际都有种苗,没有误判,穴位检出率100%.

4.   结论
  • 本文设计了一种基于机器学习的穴盘苗发芽率在线视觉检测系统,通过多视图融合可以显著地提高穴位的检出率. 首先,使用颜色空间模型对输入穴盘苗图像进行变换预处理,并归一化到统一的尺寸;然后,在预处理后的图像上制作训练集和测试集,为了提高运算速度,采用基于HOG特征和SVM分类器的方法,提取HOG特征,送入支撑向量机进行分类;最后,将单视图检测结果汇聚叠加,用多视图融合方法计算每个穴位的可信度,根据可信度判定待移栽的穴位. 试验结果表明,单视图的检出率较低,但通过多视图融合的方法不仅提高了检出率,而且有效减少了误检率和漏检率,该算法能较好地完成穴盘苗的目标检测,准确地定位发芽幼苗的穴位,同时多视图检测与传统单视图检测需要停顿相比,在实时检测的效率上有明显的改进,具有一定的应用价值.

Figure (5)  Table (1) Reference (25)

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return