4. TITLE AND SUBTITLE Real-Time Tracking and Display of Human Limb Segment Motions Using So
unit4 计算机专业英语
Notes
[5]Routing tables are the means by which a router selects the fastest or nearest path to the next “hop” on the way to a data packet’s final destination. 本句中， by which a router selects the fastest or nearest path to the next “hop” on the way to a data packet’s final destination 是一个介词前置的定语从句，修饰和限定the means。on the way to“在……途中”。
学生课下活动设计
阅读Unit4的Reading。完成作业
作业： 我的梦想是成为一家电脑公司的销售经理。 老师告诉同学们别触摸电脑屏幕。 他停下来喝了杯茶。 他努力学习为了通过期末考试。 学校是孩子们学习的好地方。 这些学生经常帮助老师打扫实验室。 昨天我忘了邮信了。 他足够大了可以自己做决定。
教学参考资料推荐
Notes
[3] The MAU does actively regenerate signals as it transmits data around the ring. 本句中，does起强调作用。 [4]Routers contain internal tables of information called routing tables that keep track of all known network addresses and possible paths throughout the internetwork,along with cost of reaching each network. 本句中，called routing tables是过去分词短语，做定语，修饰 和限定internal tables of information。that keep track of all known network addresses and possible paths throughout the internetwork,along with cost of reaching each network是一 个定语从句，修饰和限定routing tables。keep track of是“ 跟踪”。
人脸识别英文专业词汇
gallery set参考图像集Probe set=test set测试图像集face renderingFacial Landmark Detection人脸特征点检测3D Morphable Model 3D形变模型AAM (Active Appearance Model)主动外观模型Aging modeling老化建模Aging simulation老化模拟Analysis by synthesis 综合分析Aperture stop孔径光标栏Appearance Feature表观特征Baseline基准系统Benchmarking 确定基准Bidirectional relighting 双向重光照Camera calibration摄像机标定（校正）Cascade of classifiers 级联分类器face detection 人脸检测Facial expression面部表情Depth of field 景深Edgelet 小边特征Eigen light-fields本征光场Eigenface特征脸Exposure time曝光时间Expression editing表情编辑Expression mapping表情映射Partial Expression Ratio Image局部表情比率图(,PERI) extrapersonal variations类间变化Eye localization,眼睛定位face image acquisition 人脸图像获取Face aging人脸老化Face alignment人脸对齐Face categorization人脸分类Frontal faces 正面人脸Face Identification人脸识别Face recognition vendor test人脸识别供应商测试Face tracking人脸跟踪Facial action coding system面部动作编码系统Facial aging面部老化Facial animation parameters脸部动画参数Facial expression analysis人脸表情分析Facial landmark面部特征点Facial Definition Parameters人脸定义参数Field of view视场Focal length焦距Geometric warping几何扭曲Street view街景Head pose estimation头部姿态估计Harmonic reflectances谐波反射Horizontal scaling水平伸缩Identification rate识别率Illumination cone光照锥Inverse rendering逆向绘制技术Iterative closest point迭代最近点Lambertian model朗伯模型Light-field光场Local binary patterns局部二值模式Mechanical vibration机械振动Multi-view videos多视点视频Band selection波段选择Capture systems获取系统Frontal lighting正面光照Open-set identification开集识别Operating point操作点Person detection行人检测Person tracking行人跟踪Photometric stereo光度立体技术Pixellation像素化Pose correction姿态校正Privacy concern隐私关注Privacy policies隐私策略Profile extraction轮廓提取Rigid transformation刚体变换Sequential importance sampling序贯重要性抽样Skin reflectance model,皮肤反射模型Specular reflectance镜面反射Stereo baseline 立体基线Super-resolution超分辨率Facial side-view面部侧视图Texture mapping纹理映射Texture pattern纹理模式Rama Chellappa读博计划：完成先前关于指纹细节点统计建模的相关工作。
运动人体的检测跟踪方法的研究与实现
关 键 词 视频监控 运 动 检 测 运 动跟 踪 背 景 自适 应 Ｋ ｉ ｎ滤 波 ａ ｍａ
ｏＮ ＤＥＴＥＣＴＩ ＮＧ ＡＮＤ ＴＲＡＣＫＩ ＮＧ ＥＴＨ ＯＤ ｏＶＩ Ｍ ＯＦ Ｍ ＮＧ ＢｏＤＩ ＥＳ
ＡＮＤ ＴＳ ＲＥＡＵ ＺＡＴＩ Ｉ ｏＮ
人体形状 的跟 踪等 。文献 ［ ］ 二者加 以结 合 ， ３将 很好地 解决 了 多个 同向运动人体 的分 离与远处较小人体影像的跟踪 。
本文将从实验室 的环境 出发 ， 模拟外部世界 固定场景 ， 来对
机 、 以及 图像处理 、 等技术 的飞速发展 ， 网络 通信 视频监 控系统
技术也得到了相应的发展 。
差法、 背景减法。 光 流法 的优点是 能够 检测 独立 运 动的对象 ， 不需 要预先 知
１ 研 究环境 与方法
本文是在 自然光照 环境下进 行 的 ， 主要 针对 固定场景 中运
动人体 的检测与跟踪进行 系统 的研究 与实现。 系统 的硬件设 备 主要 由 Ｐ Ｃ与单 目摄 像头组 成 。其 中 ，Ｃ Ｐ
Ｔａ ｎ Ｘｉｏ Ｎａ ｆｎ ｎ Ｘｉ ａ ｎｅ ｇ （ ｃｏｌｆＣ ｍ ｕｅ Ｓｉｖ ｎ ｎ ｉ ｅｎ ，ｏｔ ｈｎ ｎ ｅｓｙｏ Ｔｃｎ ｌ ｙ Ｇ ｎ ｚｏ １ ０ ６ Ｇ ａ ｇ ｏｇ，ｈｎ ） Ｓｈｏ ｏ ｐ ｔ ｃ ｒｅａｄＥ ｇｎ ｒ ｇ ＳｕｈＣ ｉａＵ ｉ ｒｔ ｆ ｅ ｏｏ ， ｎ ｇｈｕ５ ０ ０ ， ｕ ｎｄ ｎ Ｃ ｉａ ｏ ｒ ｅｃ ｅｉ ｖ ｉ ｈ ｇ ａ
ｖｓ ｎａ ａ． ｈｓａｔｌ ｂｇ ｓｗｔ ｏ ａｋｒｕｄ ｕ ｄｔ ｇａｄｍｏｉ ｂｅｔｒｃ ｉ ｎ ｘａ ａ ｓａ ｌｏ ｔｍ ｗ ｉ ｐ ａ ｓｔ ｉｏ ｒ ｓＴ ｉ ｒｃｅ ｅｉ ｉ ｂｔ ｂ ｃｇｏｎ ｐ ａ ｎ ｎ ｖ ｇｏｊｃ ｔ ｋｎ ａｄ ｅｐｔ ｔ ｎ ａ ｒｈ ｈｃ ｕ ｄｔ ｈ ｉ ｅ ｉ ｎ ｈ ｈ ｉ ｎ ａ ｇ ｉｅ ｇｉ ｈ ｅ ｅ
人工智能原理 北京大学 4 PartIISearchingChapter4LocalSear (4.2.2)
Local Search AlgorithmsSchool of Electronic and Computer EngineeringPeking UniversityWang WenminContents☐4.2.1. Hill-Climbing Search ☐4.2.2. Local Beam Search ☐4.2.3. Tabu SearchLocal Beam Search 局部束搜索☐Keeping just one node in memory might seem to be an extreme reaction to the problem of memory limitations.在内存中仅保存一个节点似乎是对内存限制问题的极端反应。
☐Local beam search keeps track of k states rather than just 1.而局部束搜索保持k个状态而不仅仅为1。
●It begins with k randomly generated states.●At each step, all the successors of all k states are generated.●If any one is a goal, the algorithm halts, elseit selects the k best successors from the complete list, and repeats.In a local beam search, useful information can be passed among the parallel search threads.在局部束搜索中，有用的信息能够在并行搜索线程间传递。
A B C D A B C DKeeps track of k states rather than just 1. Start with k randomly generated states.k=2in this example.保持k个状态而不仅仅为1。
基于深度相机的三维人体尺寸测量
目录第一章绪论 (1)1.1 选题背景及意义 (1)1.1.1 研究目的 (1)1.1.2 研究意义 (1)1.2 国内外研究现状 (2)1.2.1 国外研究现状 (2)1.2.2 国内研究现状 (2)1.3 研究内容与方法 (4)1.3.1 研究内容 (4)1.3.2 研究方法 (5)第二章三维人体数据获取 (7)2.1 Kinect介绍 (7)2.1.1 Kinect的发展历程 (7)2.1.2 Kinect的应用 (7)2.1.3 用Kinect获取三维人体数据的优势 (8)2.1.4 Kinect二代的测量方法 (9)2.1.5 软硬件环境 (10)2.1.7 设备布局 (11)2.2 摄像机标定 (12)2.3三维人体图像融合 (12)2.4本章小结 (14)第三章三维人体点云模型 (15)3.1 点云介绍 (15)3.2点云的开发环境 (15)3.3 点云的绘制和展示 (15)3.4 相机坐标系变换到世界坐标系 (16)3.5本章小结 (17)第四章三维人体模型分割 (18)4.1 人体分割的意义 (18)4.2 利用人体区间比例进行分割的算法 (18)4.3本章小结 (20)第五章三维人体主要尺寸测量 (21)5.1 测量参数 (22)5.2 维度测量算法 (22)5.2.1 凸包算法 (22)5.2.2 躯干围度 (24)5.2.3 双臂围度 (25)5.2.4 双腿围度 (27)5.3长度测量算法 (29)5.4 本章小结 (29)第六章程序运行及误差分析 (30)6.1 程序运行 (30)6.3 本章小结 (33)第七章总结与展望 (34)7.1 全文总结 (34)7.2 展望 (34)参考文献 (36)谢辞 (38)学位论文独创性声明 (39)学位论文知识产权权属声明 (39)第一章绪论第一章绪论1.1 选题背景及意义1.1.1 研究目的Kinect[1]是微软公司研发的作为体感游戏的周边设备，本项目通过对Kinect进行开发，重点将其实施用于三维人体的测量领域。
专业英语4栏
计算机专业英语一、填空1.A bus that provides a path for moving data between system modules is called ( data bus )2. A display screen is divided intoa grid of (pixels ).3. arithmetic and logic unit executes instructions4.Another name for primary storage is (RAM )5. A Web browser is a piece of (software ).6. A bus that connects major computer components is called (system bus )7. All functions in spreadsheets start with (an equal sign ).8. An E-mail server can be considered as a(powerful operating system ) 9.A computer processes information into ( data ).10. A location in memory is accessed by its (address ).11. A small piece of code that can be transported over the Internet and executed on the recipient’s machine. The sentence describes (applet )12. A protocol is a set of (regulations ).13. A computer system has input, output, storage, and (CPU ). 14. All Intranet related documents are written in ( HTML )15. A program instruction or a piece of data is stored in a specific primary storage location called an (address )16. A collection of interconnected networks is called an (Internet ) 17. A computer system has four parts, they are output, storage, processing components and (input )18. A data model is a collection of conceptual tool for describing (data, data relationship,data semantics, data constraints ) 19. A multiprocessor system has (more than one CPU )20.CPU has only two fundamental sections: the arithmetic and logic unit and (the control unit ). 21. Data and program instructionsare stored in (memory ).22. External devices are linked toa small computer system through(interfaces ).23.. Each cell of a worksheet canhold (one piece of data ).24. External devices are linked toa small computer system through(control units ).25. HTML stand for(Hypertext Markup Language )26. hard disk is a magneticsecondary storage device27. how data are representedinside a computer system inelectronic states called (bits )28. How many basic units dotoday's digital computers consistof? ( 4 )29. LCD is based on TFT30.M emory’s contents arechanged when it is (written ).31.Once a user starts an IRC client,the server on the IRC serviceprovider side will provide the usera (channel ).32.Programs are knowncollectively as (software ).33. Programmers write (source code ).34.Primary storage provides theCPU with temporary storage forprograms and data35. The way each object combinesits member data and memberfunctions into a single structure iscalled (Encapsulation )36. The basic output device on a smallcomputer is a (display screen ).37.TheOpenSystemsInterconnection (OSI)reference model is basedon a proposal developed by ( ISO )38. The computer component thatactually manipulates the data is(the processor ).39. The physical components of acomputer are collectively called(hardware ).40. The part of an instruction thattells the processor what to do isthe (operand ).41.The permanently useful data isstored in ( the ROM )42. The interference that distortselectronic signals transmitted overa distance is called (noise ).43. To prevent user programs frominterfering with the properoperation of the system, thehardware was modified to createtwo models:(User mode and monitor mode )44. The software which acts as aninterface between a user of acomputer and the computerhardware is (operating system )45. The processor fetches andexecutes (instructions ).46. The ―brain‖ of a computersystem is (CPU )47. The ISO/OSI mode has(seven ) layers48. The source of a computer’slogic is (software ).49. The WWW is based onclient-server model standards50. The Central Processor hasonly two fundamental sections(the control unit and the arithmeticand logic unit )51. The waveform repeats thesame shape at regular intervalsand this portion is called a(period )52. WWW stand for World WideWeb二、英译汉1.class hierarchy 类层次2.XML 可扩展标记语言3.Intranet 内部网4.data bus 数据总线5.Operating system 操作系统6.CAD 计算机辅助设计7.Neural Networks 神经网络8.ODBC 开放数据库连接9.ISP 因特网服务提供商10.Expert System 专家系统11.Web site 网站站点12.WAN 广域网work communication网络通信14.FTP 文件传输协议15.system bus 系统总线16.primary memory 主存17.ROM 只读存储器18.object-oriented programming面向对象编程19.virtual reality 虚拟现实20.HTML 超文本标记语言21.Hard disk 硬盘22.database 数据库23.RAM 随机存储器三、汉译英24.主存(Main memory )25.调制解调器(modem )26.可编程只读存储器(PROM )27.数据库管理系( DBMS )28.客户端/服务器(Client/Server )29.多媒体(Multimedia )30.虚函数(virtual function )31.超文本标记语言( HTML )32.小型计算机系统接口(SCSI )33.计算机辅助软件工程(CASE )34.工具条( toolbar )35.统一资源定位符( URL )36.统一资源标识符( URI )37.人工智能( Artificial Intelligence )38.计算机( Computer )39.面向对象( Object Oriented )40.图形用户接口( Graphical user interface )41.中央处理器( CPU )42.计算机网络(computer network )43.国际互联网( Internet )44.计算机辅助制造( CAM )45.软件工程(software engineering)46.视频压缩(video compression )47.算术逻辑部件( ALU )48.计算机应用(computer application )49.电子商务(Electronic Business )50.虚拟现实(virtual reality )51.软盘驱动器( FDD )52.图形用户接口(Graphical user interface )53.应用编程接口 (API )54.视频点播VOD55.搜索引擎(search engine )56.传输控制协议/互联网协议( TCP/IP )57.万维网( WWW )58.地理信息系统( GIS )59.封装( encapsulation )60.只读存储器( ROM )61.系统软件( system software )62.磁盘操作系统(DOS )63.结构化查询语言( SQL )64.虚拟专用网( VPN )65.开放式系统互联( OSI )66.个人数字助理(PDA )67.可移植性(transportability )68.带宽(bandwidth )69.集成电路(integrated circuit )70.结构化编程(Structured programming )71.复杂指令集计算机 ( CISC )72.短信消息服务( SMS )四、判断1、ALU is one of the components of CPU.( T )2、An OS is the software which acts as an interface between a user and a computer. ( T )3、A CPU includes the ALU and the controller. ( T )4、A U-disk belongs to the output device.( T )5、A scanner belongs to the output device. ( F )6、A buffer's capacity is low and price is high, and it runs as fast asa RAM. ( F )7、A cache runs as fast as a RAM. ( F )8、Both the user names and the passwords must be unique for E-mail accounts.( F )9、Buffering is an approach to improving system performance. ( T )10、Cache memory is much faster than RAM. ( T )11、Cache memory is more expensive than RAM.( T ) 12、CD-R recorders are used to duplicate CDs ( T )13、C is an Object-Oriented programming language. ( F ) 14、Cache memory is less expensive than RAM.( F ) 15、CD-R recorders can be used to duplicate CDs( T )16、Cache memory is as fast as RAM. ( T )17、C++ language is a structured programming language.( F ) 18、CD-Rrecorders are used to duplicate CD( T )19、CPU has only one fundamental section: the control unit. ( F )20、Cache memory is much faster than RAM. ( T )21、CD-ROM stands for compact disk read-only memory. ( T ) 22、DVD provides CPU with temporary storage for programs and data. (F )23、E-mail is a method of sending and receiving messages on the Internet .( F )24、FTP is a protocol of the computer network. ( T )25、FTP is not a protocol of the computer network. ( F )26、HTTP stands for High Transportation Port. ( F ) 27、Hard disk provides CPU withtemporary storage for programsand data. (F )28、HTTP is used to locateInternet resources. ( F )29、HTML is used to write Webpages.( T)30、Hard disks and floppiesoperate in different ways. ( F )31、Internet resources are storedon Web servers. ( T )32、JA V A is a Object-Orientedprogramming language. ( T )33、JA V A is a structuredprogramming language. ( F )34、Keyboard and mouse are boththe input devices of the computer.( T )35、Keyboard and mouse are boththe input devices of the computer.( T )36、Linux is an operating system.( T )37、Microsoft SQL server 2000 isan operating system of thecomputer. ( F )38、Microsoft Office 2003 isoperating system software. ( F )39、Microsoft Office 2003 is anoperating system software. ( F )40、Microsoft Windows 2000 is anoperating system of the computer.( T )41、MS-DOSis a Object-Orientedprogramming language. ( F )42、MS Windows 2000 is ahardware component of computer.( F )43、MS Office 2000 is the firstWindows operating system in areal sense. ( F )44、MS-DOS refers to MicrosoftDisk Operating System. ( T )45、OS is the software which actsas an interface between a user anda computer. ( T )46、PROM is the abbreviation ofportable read –only memory.( F )47、Printer is the input device ofthe computer. ( F )48、PCI is a popularlow-bandwidth bus. (F )49、Primary storage provides CPUwith temporary storage forprograms and data. (T)50、Printer is the I/O device of thecomputer. ( T )51、PDA is one of the componentsof CPU. ( F )52、PCI is a popularlow-bandwidth bus. ( F )53、ROM is the abbreviation ofread of memory. ( F )54、PCI is abbreviation ofperipheral componentinterconnect.( T )55、System bus provides a path formoving data between systemmodules. ( F )56、The system clock is the brainof a computer. ( F )57、The system clock sends outpulses regularly. ( T )58、There 3 main categories ofoptical laser disks: CD-ROM,DVD and VCD. ( F )59、The Web browser is used todisplay the data stored on theInternet. ( T )60、The wide of the bus is alsocalled the "word length". ( T )61、The user name must be uniquefor E-mail accounts. ( T )62、The single-sided DVD canstore 6.5GB of data. ( F )63、To format a floppy means thatmagnetic areas are created.( T )64、The CPU is the brain of acomputer( T )65、URL is used to locate Internetresources. ( T )66、VDT stands for video displayterminal. ( T )67、WWW stands for World WideWeb. ( T )68、Windows 2000 is the firstWindows operating system in areal sense.( F )69、We can view an operatingsystem as a resource manager.( T )70、We can view an operatingsystem as a output hardware.( T )71、We can view an OS as aresource manager. ( T )72、Windows 95 is the firstWindows operating system in areal sense. ( T )73、XML is used to write Webpages.( F )74、You can connect 120 devicesto a computer by USB. ( T)75、You can connect 127 devicesto a computer by USB. ( T )76、You can connect 255 devicesto a computer by USB. ( F )五、翻译1.A handheld’s constraints（局限性）mean that it’s usuallyimpossible to directly port adesktop application to a wirelesshandheld device. For the samereasons, it’s difficult to directlyaccess most Web sites with ahandheld device. Web applicationsare traditionally designed based onthe assumption that visitors willhave a desktop computer with alarge screen and a mouse.手持设备的局限性意味着它通常不能直接把台式应用程序移植到无线手持设备上，出于同样的理由，用手持设备直接访问多数Web站点也很困难。
弱监督任意姿态人体检测
弱监督任意姿态人体检测*蔡雅薇+，谭晓阳南京航空航天大学计算机科学与技术学院，南京210016Weakly Supervised Human Body Detection under Arbitrary PosesCAI Yawei +,TAN XiaoyangCollege of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 210016,China+Corresponding author:E-mail:caiyawei@CAI Y awei,TAN Xiaoyang.Weakly supervised human body detection under arbitrary poses.Journal of Frontiers of Computer Science and Technolog y ,2017,11(4)：587-598.Abstract:The problem of weakly supervised human body detection under difficult poses (e.g.,multi-view and/or arbitrary poses)is studied.Most current methods on human body detection focuse only on a few common human body poses with human body in upright positions,while in the real world human bodies may exhibit very rich pose variations (e.g.,when people are bending,sleeping or sitting).This not only imposes great challenges on the task of human detection,but also makes the job of manual annotation even more difficult,and usually only weak annota-tions are available in practice.The multiple instance learning method relaxes the requirements of accurate labeling and hence is commonly used to address the task.However,it is sensitive to the quality of positive instances and the settings of some model parameters such as the strategy to fuse the instance-level conditional probability into a bag-level one.This paper presents a comprehensive and in-depth empirical method of these important but less studied issues on the person dataset of Pascal VOC 2007,and proposes a new selective weakly supervised detection algo-rithm (SWSD).Experiments demonstrate that with only a few fully supervised samples,the performance of weakly supervised human body detection can be significantly improved under the multiple instance learning framework.Key words:weakly supervision;human body detection;arbitrary poses;multiple instance learning*The National Natural Science Foundation of China under Grant No.61373060(国家自然科学基金).Received 2016-03,Accepted 2016-05.CNKI 网络优先出版:2016-05-13,/kcms/detail/11.5602.TP.20160513.1434.004.htmlISSN 1673-9418CODEN JKYTA8Journal of Frontiers of Computer Science and Technology 1673-9418/2017/11(04)-0587-12doi:10.3778/j.issn.1673-9418.1603044E-mail:fcst@ Tel:+86-10-89056056Journal of Frontiers of Computer Science and Technology 计算机科学与探索2017,11(4)1引言多姿态人体检测在日常生活中有很多重要的应用[1]。
toc
Table of ContentsPreface .................................................................................................................................................................... x iii Organization ......................................................................................................................................................... x viii Program Committees ............................................................................................................................................ x ix Funding ................................................................................................................................................................. x xiii Invited Talks .......................................................................................................................................................... x xv Tutorial and Workshop Summaries ................................................................................................................. x xxiii The Role of Machine Learning in Business Optimization (1)Chid ApteFAB-MAP: Appearance-Based Place Recognition and Mapping using a Learned VisualVocabulary Model (3)Mark Cummins, Paul NewmanDiscriminative Latent Variable Models for Object Detection (11)Pedro Felzenszwalb, Ross Girshick, David McAllester, Deva RamananWeb-Scale Bayesian Click-Through rate Prediction for Sponsored Search Advertising inMicrosoft's Bing Search Engine (13)Thore Graepel, Joaquin Quiñonero Candela, Thomas Borchert, Ralf HerbrichMusic Plus One and Machine Learning (21)Christopher RaphaelClimbing the Tower of Babel: Unsupervised Multilingual Learning (29)Benjamin Snyder, Regina BarzilayDetecting Large-Scale System Problems by Mining Console Logs (37)Wei Xu, Ling Huang, Armando Fox, David Patterson, Michael I. JordanParticle Filtered MCMC-MLE with Connections to Contrastive Divergence (47)Arthur U. Asuncion, Qiang Liu, Alexander T. Ihler, Padhraic SmythSurrogating the surrogate: accelerating Gaussian-process-based global optimization with amixture cross-entropy algorithm (55)Remi Bardenet, Balazs KeglForgetting Counts : Constant Memory Inference for a Dependent Hierarchical Pitman-YorProcess (63)Nicholas Bartlett, David Pfau, Frank WoodRobust Formulations for Handling Uncertainty in Kernel Matrices (71)Sahely Bhadra, Sourangshu Bhattacharya, Chiranjib Bhattacharyya, Aharon Ben-talActive Learning for Networked Data (79)Mustafa Bilgic, Lilyana Mihalkova, Lise GetoorDistance dependent Chinese restaurant processes (87)David M. Blei, Peter FrazierCausal filter selection in microarray data (95)Gianluca Bontempi, Patrick E. MeyerLabel Ranking under Ambiguous Supervision for Learning Semantic Correspondences (103)Antoine Bordes, Nicolas Usunier, Jason WestonA Theoretical Analysis of Feature Pooling in Visual Recognition (111)Y-Lan Boureau, Jean Ponce, Yann LeCunMulti-agent Learning Experiments on Repeated Matrix Games (119)Bruno Bouzy, Marc MetivierLearning Tree Conditional Random Fields (127)Joseph K. Bradley, Carlos GuestrinFinding Planted Partitions in Nearly Linear Time using Arrested Spectral Clustering (135)Nader H. Bshouty, Philip M. LongFast boosting using adversarial bandits (143)Róbert Busa-Fekete, Balázs KéglModeling Transfer Learning in Human Categorization with the Hierarchical Dirichlet Process (151)Kevin R. Canini, Mikhail M. Shashkov, Thomas L. GriffithsTransfer Learning for Collective Link Prediction in Multiple Heterogenous Domains (159)Bin Cao, Nathan Nan Liu, Qiang YangThe Elastic Embedding Algorithm for Dimensionality Reduction (167)Miguel Á. Carreira-PerpiñánRandom Spanning Trees and the Prediction of Weighted Graphs (175)Nicolo Cesa-Bianchi, Claudio Gentile, Fabio Vitale, Giovanni ZappellaEfficient Learning with Partially Observed Attributes (183)Nicolo Cesa-Bianchi, Shai Shalev-Shwartz, Ohad ShamirConvergence, Targeted Optimality, and Safety in Multiagent Learning (191)Doran Chakraborty, Peter StoneStructured Output Learning with Indirect Supervision (199)Ming-Wei Chang, Vivek Srikumar, Dan Goldwasser, Dan RothDynamical Products of Experts for Modeling Financial Time Series (207)Yutian Chen, Max WellingLabel Ranking Methods based on the Plackett-Luce Model (215)Weiwei Cheng, Krzysztof Dembczynski, Eyke HüllermeierGraded Multilabel Classification: The Ordinal Case (223)Weiwei Cheng, Krzysztof Dembczynski, Eyke HüllermeierComparing Clusterings in Space (231)Michael H. Coen, M. Hidayath Ansari, Nathanael FillmoreTwo-Stage Learning Kernel Algorithms (239)Corinna Cortes, Mehryar Mohri, Afshin RostamizadehGeneralization Bounds for Learning Kernels (247)Corinna Cortes, Mehryar Mohri, Afshin RostamizadehFast Neighborhood Subgraph Pairwise Distance Kernel (255)Fabrizio Costa, Kurt De GraveMining Clustering Dimensions (263)Sajib Dasgupta, Vincent NgBottom-Up Learning of Markov Network Structure (271)Jesse Davis, Pedro DomingosBayes Optimal Multilabel Classification via Probabilistic Classifier Chains (279)Krzysztof Dembczynski, Weiwei Cheng, Eyke HüllermeierA Conditional Random Field for Multiple-Instance Learning (287)Thomas Deselaers, Vittorio FerrariAsymptotic Analysis of Generative Semi-Supervised Learning (295)Joshua V. Dillon, Krishnakumar Balasubramanian, Guy LebanonHeterogeneous Continuous Dynamic Bayesian Networks with Flexible Structure and Inter-TimeSegment Information Sharing (303)Frank Dondelinger, Sophie Lebre, Dirk HusmeierTemporal Difference Bayesian Model Averaging:A Bayesian Perspective on Adapting Lambda (311)Carlton Downey, Scott SannerHigh-Performance Semi-Supervised Learning using Discriminatively Constrained GenerativeModels (319)Gregory Druck, Andrew McCallumOn the Consistency of Ranking Algorithms (327)John C. Duchi, Lester W. Mackey, Michael I. JordanInverse Optimal Control with Linearly-Solvable MDPs (335)Krishnamurthy Dvijotham, Emanuel TodorovContinuous-Time Belief Propagation (343)Tal El-Hay, Ido Cohn, Nir Friedman, Raz KupfermanNonparametric Information Theoretic Clustering Algorithm (351)Lev Faivishevsky, Jacob GoldbergerFeature Selection as a One-Player Game (359)Romaric Gaudel, Michele SebagMultiscale Wavelets on Trees, Graphs and High Dimensional Data: Theory and Applications toSemi Supervised Learning (367)Matan Gavish, Boaz Nadler, Ronald R. CoifmanA Language-based Approach to Measuring Scholarly Impact (375)Sean M. Gerrish, David M. BleiBoosting Classifiers with Tightened L0-Relaxation Penalties (383)Noam Goldberg, Jonathan EcksteinBudgeted Nonparametric Learning from Data Streams (391)Ryan Gomes, Andreas KrauseLearning Fast Approximations of Sparse Coding (399)Karol Gregor, Yann LeCunBoosted Backpropagation Learning for Training Deep Modular Networks (407)Alexander Grubb, J. Andrew BagnellInteractive Submodular Set Cover (415)Andrew Guillory, Jeff BilmesLarge Scale Max-Margin Multi-Label Classification with Priors (423)Bharath Hariharan, Lihi Zelnik-Manor, S. V. N. Vishwanathan, Manik VarmaActive Learning for Multi-Task Adaptive Filtering (431)Abhay Harpale, Yiming YangBayesian Nonparametric Matrix Factorization for Recorded Music (439)Matthew D. Hoffman, David M. Blei, Perry R. CookMulti-Task Learning of Gaussian Graphical Models (447)Jean Honorio, Dimitris SamarasLearning Hierarchical Riffle Independent Groupings from Rankings (455)Jonathan Huang, Carlos GuestrinOn learning with kernels for unordered pairs (463)Martial Hue, Jean-Philippe VertA Simple Algorithm for Nuclear Norm Regularized Problems (471)Martin Jaggi, Marek SulovskýTelling cause from effect based on high-dimensional observations (479)Dominik Janzing, Patrik O. Hoyer, Bernhard SchölkopfProximal Methods for Sparse Hierarchical Dictionary Learning (487)Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, Francis Bach3D Convolutional Neural Networks for Human Action Recognition (495)Shuiwang Ji, Wei Xu, Ming Yang, Kai YuAccelerated dual decomposition for MAP inference (503)Vladimir Jojic, Stephen Gould, Daphne KollerEfficient Selection of Multiple Bandit Arms: Theory and Practice (511)Shivaram Kalyanakrishnan, Peter StoneA scalable trust-region algorithm with application to mixed-norm regression (519)Dongmin Kim, Suvrit Sra, Inderjit DhillonLocal Minima Embedding (527)Minyoung Kim, Fernando De la TorreGaussian Processes Multiple Instance Learning (535)Minyoung Kim, Fernando De la TorreTree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity (543)Seyoung Kim, Eric P. XingLearning Markov Logic Networks Using Structural Motifs (551)Stanley Kok, Pedro DomingosOn Sparse Nonparametric Conditional Covariance Selection (559)Mladen Kolar, Ankur P. Parikh, Eric P. XingSubmodular Dictionary Selection for Sparse Representation (567)Andreas Krause, Volkan CevherImplicit Online Learning (575)Brian Kulis, Peter L. BartlettProbabilistic Backward and Forward Reasoning in Stochastic Relational Worlds (583)Tobias Lang, Marc ToussaintSupervised Aggregation of Classifiers using Artificial Prediction Markets (591)Nathan Lay, Adrian BarbuBayesian Multi-Task Reinforcement Learning (599)Alessandro Lazaric, Mohammad GhavamzadehAnalysis of a Classification-based Policy Iteration Algorithm (607)Alessandro Lazaric, Mohammad Ghavamzadeh, Rémi MunosFinite-Sample Analysis of LSTD (615)Alessandro Lazaric, Mohammad Ghavamzadeh, Rémi MunosA fast natural Newton method (623)Nicolas Le Roux, Andrew FitzgibbonMaking Large-Scale Nystr•om Approximation Possible (631)Mu Li, James T. Kwok, Bao-Liang LuLearning Programs: A Hierarchical Bayesian Approach (639)Percy Liang, Michael I. Jordan, Dan KleinOn the Interaction between Norm and Dimensionality: Multiple Regimes in Learning (647)Percy Liang, Nati SrebroPower Iteration Clustering (655)Frank Lin, William W. CohenRobust Subspace Segmentation by Low-Rank Representation (663)Guangcan Liu, Zhouchen Lin, Yong YuRobust Graph Mode Seeking by Graph Shift (671)Hairong Liu, Shuicheng YanLarge Graph Construction for Scalable Semi-Supervised Learning (679)Wei Liu, Junfeng He, Shih-Fu ChangLearning Temporal Causal Graphs for Relational Time-Series Analysis (687)Yan Liu, Alexandru Niculescu-Mizil, Aurélie Lozano , Yong LuEfficient Reinforcement Learning with Multiple Reward Functions for Randomized ControlledTrial Analysis (695)Daniel J. Lizotte, Michael Bowling, Susan A. MurphyRestricted Boltzmann Machines are Hard to Approximately Evaluate or Simulate (703)Philip M. Long, Rocco A. ServedioMixed Membership Matrix Factorization (711)Lester Mackey, David Weiss, Michael I. JordanToward Off-Policy Learning Control with Function Approximation (719)Hamid Reza Maei, Csaba Szepesvari, Shalabh Bhatnagar, Richard S. SuttonConstructing States for Reinforcement Learning (727)M. M. Hassan MahmudDeep learning via Hessian-free optimization (735)James MartensLearning the Linear Dynamical System with ASOS (743)James MartensFrom Transformation-Based Dimensionality Reduction to Feature Selection (751)Mahdokht Masaeli, Glenn Fung, Jennifer G. DyRisk minimization, probability elicitation, and cost-sensitive SVMs (759)Hamed Masnadi-Shirazi, Nuno VasconcelosExploiting Data-Independence for Fast Belief-Propagation (767)Julian J. McAuley, Tibério S. CaetanoMetric Learning to Rank (775)Brian McFee, Gert LanckrietLearning Efficiently with Approximate Inference via Dual Losses (783)Ofer Meshi, David Sontag, Tommi Jaakkola, Amir GlobersonDeep Supervised t-Distributed Embedding (791)Renqiang Min, Laurens van der Maaten, Zineng Yuan, Anthony Bonner, Zhaolei ZhangNonparametric Return Distribution Approximation for Reinforcement Learning (799)Tetsuro Morimura, Masashi Sugiyama, Hisashi Kashima , Hirotaka Hachiya, Toshiyuki TanakaRectified Linear Units Improve Restricted Boltzmann Machines (807)Vinod Nair, Geoffrey E. HintonImplicit Regularization in Variational Bayesian Matrix Factorization (815)Shinichi Nakajima, Masashi SugiyamaEstimation of (near) low-rank matrices with noise and high-dimensional scaling (823)Sahand Negahban, Martin J. WainwrightMultiple Non-Redundant Spectral Clustering Views (831)Donglin Niu, Jennifer G. Dy, Michael I. JordanMultiagent Inductive Learning: an Argumentation-based Approach (839)Santiago Ontañón, Enric PlazaA Stick-Breaking Construction of the Beta Process (847)John Paisley, Aimee Zaas, Christopher W. Woods, Geoffrey S. Ginsburg, Lawrence CarinThe Margin Perceptron with Unlearning (855)Constantinos Panagiotakopoulos, Petroula TsampoukaBoosting for Regression Transfer (863)David Pardoe, Peter StoneFeature Selection Using Regularization in Approximate Linear Programs for Markov DecisionProcesses (871)Marek Petrik, Gavin Taylor, Ron Parr, Shlomo Zilberstein*Budgeted Distribution Learning of Belief Net Parameters (879)Liuyang Li, Barnabás Póczos, Csaba Szepesvári, Russ GreinerVariable Selection in Model-Based Clustering: To Do or To Facilitate (887)Leonard K. M. Poon, Nevin L. Zhang, Tao Chen, Yi WangApproximate Predictive Representations of Partially Observable Systems (895)Monica Dinculescu, Doina PrecupSpherical Topic Models (903)Joseph Reisinger, AustinWaters, Bryan Silverthorn, Raymond J. MooneySVM Classifier Estimation from Group Probabilities (911)Stefan RuepingClustering processes (919)Daniil RyabkoGaussian Process Change Point Models (927)Yunus Saatçi, Ryan Turner, Carl Edward RasmussenOnline Prediction with Privacy (935)Jun Sakuma, Hiromi AraiLearning Deep Boltzmann Machines using Adaptive MCMC (943)Ruslan SalakhutdinovActive Risk Estimation (951)Christoph Sawade , Niels Landwehr, Steffen Bickel, Tobias SchefferShould one compute the Temporal Difference fix point or minimize the Bellman Residual ? Theunified oblique projection view (959)Bruno ScherrerGaussian Covariance and Scalable Variational Inference (967)Matthias W. SeegerApplication of Machine Learning To Epileptic Seizure Detection (975)Ali Shoeb, John GuttagLearning optimally diverse rankings over large document collections (983)Aleksandrs Slivkins, Filip Radlinski, Sreenivas GollapudiHilbert Space Embeddings of Hidden Markov Models (991)Le Song, Sajid M. Siddiqi, Geoffrey Gordon, Alex SmolaCOFFIN : A Computational Framework for Linear SVMs (999)Soeren Sonnenburg, Vojtech FrancInternal Rewards Mitigate Agent Boundedness (1007)Jonathan Sorg, Satinder Singh, Richard LewisGaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design (1015)Niranjan Srinivas, Andreas Krause, Sham Kakade, Matthias SeegerUnsupervised Risk Strati cation in Clinical Datasets: Identifying Patients at Risk of RareOutcomes (1023)Zeeshan Syed, Ilan RubinfeldModel-based reinforcement learning with nearly tight exploration complexity bounds (1031)István Szita, Csaba SzepesváriTotal Variation and Cheeger Cuts (1039)Arthur Szlam, Xavier BressonLearning Sparse SVM for Feature Selection on Very High Dimensional Datasets (1047)Mingkui Tan, Li Wang, Ivor W. TsangDeep networks for robust visual recognition (1055)Yichuan Tang, Chris EliasmithA DC Programming Approach for Sparse Eigenvalue Problem (1063)Mamadou Thiao, Tao Pham Dinh, Hoai An Le ThiLeast-Squares Policy Iteration: Bias-Variance Trade-off in Control Problems (1071)Christophe Thiery, Bruno ScherrerAn Analysis of the Convergence of Graph Laplacians (1079)Daniel Ting, Ling Huang, Michael I. JordanA Fast Augmented Lagrangian Algorithm for Learning Low-Rank Matrices (1087)Ryota Tomioka, Taiji Suzuki, Masashi Sugiyama, Hisashi KashimaOne-sided Support Vector Regression for Multiclass Cost-sensitive Classification (1095)Han-Hsing Tu, Hsuan-Tien LinNon-Local Contrastive Objectives (1103)David Vickrey, Cliff Chiung-Yu Lin, Daphne KollerThe Translation-invariant Wishart-Dirichlet Process for Clustering Distance Data (1111)Julia E. Vogt, Sandhya Prabhakaran, Thomas J. Fuchs, Volker RothGeneralizing Apprenticeship Learning across Hypothesis Classes (1119)Thomas J. Walsh, Kaushik Subramanian, Michael L. Littman, Carlos DiukSequential Projection Learning for Hashing with Compact Codes (1127)Jun Wang, Sanjiv Kumar, Shih-Fu ChangA New Analysis of Co-Training (1135)Wei Wang, Zhi-Hua ZhouMulti-Class Pegasos on a Budget (1143)Zhuang Wang, Koby Crammer, Slobodan VuceticThe IBP Compound Dirichlet Process and its Application to Focused Topic Modeling (1151)Sinead Williamson, Chong Wang, Katherine A. Heller, David M. BleiOnline Streaming Feature Selection (1159)Xindong Wu, Kui Yu, Hao Wang, Wei DingClasses of Multiagent Q-learning Dynamics with -greedy Exploration (1167)Michael Wunder, Michael Littman, Monica BabesSimple and Efficient Multiple Kernel Learning by Group Lasso (1175)Zenglin Xu, Rong Jin, Haiqin Yang, Irwin King, Michael R. LyuSparse Gaussian Process Regression via l1 Penalization (1183)Feng Yan, Yuan (Alan) QiOnline Learning for Group Lasso (1191)Haiqin Yang, Zenglin Xu, Irwin King, Michael R. LyuLearning from Noisy Side Information by Generalized Maximum Entropy Model (1199)Tianbao Yang, Rong Jin, Anil K. JainConvergence of Least Squares Temporal Difference Methods Under General Conditions (1207)Huizhen YuImproved Local Coordinate Coding using Local Tangents (1215)Kai Yu, Tong ZhangProjection Penalties: Dimension Reduction without Loss (1223)Yi Zhang, Jeff SchneiderOTL: A Framework of Online Transfer Learning (1231)Peilin Zhao, Steven C.H. HoiConditional Topic Random Fields (1239)Jun Zhu, Eric P. XingCognitive Models of Test-Item Effects in Human Category Learning (1247)Xiaojin Zhu, Bryan R. Gibson, Kwang-Sung Jun, Timothy T. Rogers, Joseph Harrison, Chuck KalishModeling Interaction via the Principle of Maximum Causal Entropy (1255)Brian D. Ziebart, J. Andrew Bagnell, Anind K. Dey。
openpose模型在AIchallenge人体骨骼关键点检测的表现
openpose模型在AIchallenge⼈体⾻骼关键点检测的表现因为之前正好看了CMU在CVPR2017上的论⽂《Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields》，⽽且他们提供了训练好的模型。
所以就直接⽤CMU训练的模型在AI challenge的数据集上做了测试。
最后没有使⽤AI challenge训练集训练的模型在AI challenge上的得分是0.1667，可以看作是⼀个baseline。
以下是预处理的说明以及加⼊预处理程序的源代码。
openpose的源代码使⽤#openpose ##openpose标注出来了，剩下的就是AI challenge的预处理程序。
在Google Cloud 上使⽤1⽚NVIDIA Tesla K80 跑完AI challenge的测试集⼤约需要24⼩时，4秒左右处理⼀副图。
AI challenge测试要求的关键点顺序是：1右肩，2右肘，3右腕，4左肩，5左肘，6左腕，7右髋，8右膝，9右踝，10左髋，11左膝，12左踝，13头顶，14脖⼦openpose源码中subset输出的关键点顺序是：1⿐⼦，2脖⼦，3右肩，4右肘，5右腕，6左肩，7左肘，8左腕，9右髋，10右膝，11右踝，12左髋，13左膝，14左踝，15左眼，16右眼，17左⽿，18右⽿，19 pt19函数 subset2AIsubset, all_peaks2all_peaks_1d, listMultiKeypoints 负责把openpose的关键点转换成AI challenge 的关键点。
当然还得按照官⽹上的要求输出特定格式的JSON⽂件,如下所⽰：[{"image_id": "a0f6bdc065a602b7b84a67fb8d14ce403d902e0d","keypoint_annotations": {"human1": [261, 294, 1, 281, 328, 1, 0, 0, 0, 213, 295, 1, 208, 346, 1, 192, 335, 1, 245, 375, 1, 255, 432, 1, 244, 494, 1, 221, 379, 1, 219, 442, 1, 226, 491, 1, 226, 256, 1, 231, 284, 1], "human2": [313, 301, 1, 305, 337, 1, 321, 345, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 313, 359, 1, 320, 409, 1, 311, 454, 1, 0, 0, 0, 330, 409, 1, 324, 446, 1, 337, 284, 1, 327, 302, 1],"human3": [373, 304, 1, 346, 286, 1, 332, 263, 1, 0, 0, 0, 0, 0, 0, 345, 313, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 363, 386, 1, 361, 424, 1, 361, 475, 1, 365, 273, 1, 369, 297, 1],...}}...]1#import numpy as np2import json3import os4#openpose5import keras6from keras.models import Sequential7from keras.models import Model8from yers import Input, Dense, Activation9from yers.convolutional import Conv2D10from yers.pooling import MaxPooling2D11from yers.normalization import BatchNormalization12from yers.merge import Concatenate13from config_reader import config_reader14import scipy1516import cv217import numpy as np18 np.seterr(divide='ignore', invalid='ignore')19import util20import math21from numpy import ma22from scipy.ndimage.filters import gaussian_filter23##openpose24#openpose25def relu(x):26return Activation('relu')(x)2728def conv(x, nf, ks, name):29 x1 = Conv2D(nf, (ks, ks), padding='same', name=name)(x)30return x13132def pooling(x, ks, st, name):33 x = MaxPooling2D((ks, ks), strides=(st, st), name=name)(x)34return x3536def vgg_block(x):3738# Block 139 x = conv(x, 64, 3, "conv1_1")40 x = relu(x)41 x = conv(x, 64, 3, "conv1_2")42 x = relu(x)43 x = pooling(x, 2, 2, "pool1_1")4445# Block 246 x = conv(x, 128, 3, "conv2_1")47 x = relu(x)48 x = conv(x, 128, 3, "conv2_2")49 x = relu(x)50 x = pooling(x, 2, 2, "pool2_1")5152# Block 353 x = conv(x, 256, 3, "conv3_1")54 x = relu(x)55 x = conv(x, 256, 3, "conv3_2")56 x = relu(x)57 x = conv(x, 256, 3, "conv3_3")59 x = conv(x, 256, 3, "conv3_4")60 x = relu(x)61 x = pooling(x, 2, 2, "pool3_1")6263# Block 464 x = conv(x, 512, 3, "conv4_1")65 x = relu(x)66 x = conv(x, 512, 3, "conv4_2")67 x = relu(x)6869# Additional non vgg layers70 x = conv(x, 256, 3, "conv4_3_CPM")71 x = relu(x)72 x = conv(x, 128, 3, "conv4_4_CPM")73 x = relu(x)7475return x7677def stage1_block(x, num_p, branch):7879# Block 180 x = conv(x, 128, 3, "conv5_1_CPM_L%d" % branch)81 x = relu(x)82 x = conv(x, 128, 3, "conv5_2_CPM_L%d" % branch)83 x = relu(x)84 x = conv(x, 128, 3, "conv5_3_CPM_L%d" % branch)85 x = relu(x)86 x = conv(x, 512, 1, "conv5_4_CPM_L%d" % branch)87 x = relu(x)88 x = conv(x, num_p, 1, "conv5_5_CPM_L%d" % branch)8990return x9192def stageT_block(x, num_p, stage, branch):9394# Block 195 x = conv(x, 128, 7, "Mconv1_stage%d_L%d" % (stage, branch))96 x = relu(x)97 x = conv(x, 128, 7, "Mconv2_stage%d_L%d" % (stage, branch))98 x = relu(x)99 x = conv(x, 128, 7, "Mconv3_stage%d_L%d" % (stage, branch)) 100 x = relu(x)101 x = conv(x, 128, 7, "Mconv4_stage%d_L%d" % (stage, branch)) 102 x = relu(x)103 x = conv(x, 128, 7, "Mconv5_stage%d_L%d" % (stage, branch)) 104 x = relu(x)105 x = conv(x, 128, 1, "Mconv6_stage%d_L%d" % (stage, branch)) 106 x = relu(x)107 x = conv(x, num_p, 1, "Mconv7_stage%d_L%d" % (stage, branch)) 108109return x110##openpose111112def subset2AIsubset(t, numPersons):113 AIsubset=[]114for j in xrange(numPersons):115 tempsubset=[]116for i in xrange(12):117#20118#print(i+2)119 tempsubset.append(t[j][i+2])120121 tempsubset.append(t[j][0])122 tempsubset.append(t[j][1])123#print(AIsubset)124 AIsubset.append(tempsubset)125return AIsubset126127def all_peaks2all_peaks_1d(all_peaks):128 all_peaks_1d=[]129for item in all_peaks:130for item1 in item:131 all_peaks_1d.append(item1)132return all_peaks_1d133134def listMultiKeypoints(all_peaks_1d, numPersons):135 multi_keypoints=[]136for i in xrange(numPersons):137 sp_keypoints=[]138for j in xrange(14):139if(AIsubset[i][j]== -1.):140 sp_keypoints.append(0)141 sp_keypoints.append(0)142 sp_keypoints.append(0)143else:144 sp_keypoints.append(all_peaks_1d[int(AIsubset[i][j])][0]) 145 sp_keypoints.append(all_peaks_1d[int(AIsubset[i][j])][1]) 146 sp_keypoints.append(1)147#print(sp_keypoints)148 multi_keypoints.append(sp_keypoints)149return multi_keypoints150152return len(t)153154def listHuman(nPersons):155 list_human=[]156for i in xrange(numPersons):157 list_human.append('human'+str(i+1))158return list_human159160161162#openpose163 weights_path = "model/keras/model.h5"164165 input_shape = (None,None,3)166167 img_input = Input(shape=input_shape)168169 stages = 6170 np_branch1 = 38171 np_branch2 = 19172173# VGG174 stage0_out = vgg_block(img_input)175176# stage 1177 stage1_branch1_out = stage1_block(stage0_out, np_branch1, 1)178 stage1_branch2_out = stage1_block(stage0_out, np_branch2, 2)179 x = Concatenate()([stage1_branch1_out, stage1_branch2_out, stage0_out])180181# stage t >= 2182for sn in range(2, stages + 1):183 stageT_branch1_out = stageT_block(x, np_branch1, sn, 1)184 stageT_branch2_out = stageT_block(x, np_branch2, sn, 2)185if (sn < stages):186 x = Concatenate()([stageT_branch1_out, stageT_branch2_out, stage0_out])187188 model = Model(img_input, [stageT_branch1_out, stageT_branch2_out])189 model.load_weights(weights_path)190##openpose191192#openpose193# find connection in the specified sequence, center 29 is in the position 15194 limbSeq = [[2,3], [2,6], [3,4], [4,5], [6,7], [7,8], [2,9], [9,10], \195 [10,11], [2,12], [12,13], [13,14], [2,1], [1,15], [15,17], \196 [1,16], [16,18], [3,17], [6,18]]197# the middle joints heatmap correpondence198 mapIdx = [[31,32], [39,40], [33,34], [35,36], [41,42], [43,44], [19,20], [21,22], \199 [23,24], [25,26], [27,28], [29,30], [47,48], [49,50], [53,54], [51,52], \200 [55,56], [37,38], [45,46]]201##openpose202203 path = "./test0"204 files = os.listdir(path)205 list_image_names=[]206 final_results=[]207 num_processed_images=0.208 total_images=30000.209for file in files:210 num_processed_images+=1211print('file:',file)212print('number of image:',num_processed_images)213print('%.2f%%'%(num_processed_images/total_images*100))214 list_image_names.append(str(file)[:-4])215#openpose216 test_image = './test0/'+file217#test_image = 'sample_images/000a902c8674739c97f188157c63d709b45b7595.jpg'218 oriImg = cv2.imread(test_image)219220 param, model_params = config_reader()221 multiplier = [x * model_params['boxsize'] / oriImg.shape[0] for x in param['scale_search']]222 heatmap_avg = np.zeros((oriImg.shape[0], oriImg.shape[1], 19))223 paf_avg = np.zeros((oriImg.shape[0], oriImg.shape[1], 38))224225for m in range(len(multiplier)):226 scale = multiplier[m]227 imageToTest = cv2.resize(oriImg, (0,0), fx=scale, fy=scale, interpolation=cv2.INTER_CUBIC)228 imageToTest_padded, pad = util.padRightDownCorner(imageToTest, model_params['stride'], model_params['padValue'])229230 input_img = np.transpose(np.float32(imageToTest_padded[:,:,:,np.newaxis]), (3,0,1,2))/256 - 0.5; # required shape (1, width, height, channels) 231print("Input shape: " + str(input_img.shape))232233 output_blobs = model.predict(input_img)234print("Output shape (heatmap): " + str(output_blobs[1].shape))235236# extract outputs, resize, and remove padding237 heatmap = np.squeeze(output_blobs[1]) # output 1 is heatmaps238 heatmap = cv2.resize(heatmap, (0,0), fx=model_params['stride'], fy=model_params['stride'], interpolation=cv2.INTER_CUBIC)239 heatmap = heatmap[:imageToTest_padded.shape[0]-pad[2], :imageToTest_padded.shape[1]-pad[3], :]240 heatmap = cv2.resize(heatmap, (oriImg.shape[1], oriImg.shape[0]), interpolation=cv2.INTER_CUBIC)241242 paf = np.squeeze(output_blobs[0]) # output 0 is PAFs243 paf = cv2.resize(paf, (0,0), fx=model_params['stride'], fy=model_params['stride'], interpolation=cv2.INTER_CUBIC)244 paf = paf[:imageToTest_padded.shape[0]-pad[2], :imageToTest_padded.shape[1]-pad[3], :]245 paf = cv2.resize(paf, (oriImg.shape[1], oriImg.shape[0]), interpolation=cv2.INTER_CUBIC)246247 heatmap_avg = heatmap_avg + heatmap / len(multiplier)248 paf_avg = paf_avg + paf / len(multiplier)249##openpose250#openpose251 U = paf_avg[:,:,16] * -1252 V = paf_avg[:,:,17]253 X, Y = np.meshgrid(np.arange(U.shape[1]), np.arange(U.shape[0]))254 M = np.zeros(U.shape, dtype='bool')255 M[U**2 + V**2 < 0.5 * 0.5] = True256 U = ma.masked_array(U, mask=M)257 V = ma.masked_array(V, mask=M)258259260 all_peaks = []261 peak_counter = 0262263for part in range(19-1):264 map_ori = heatmap_avg[:,:,part]265 map = gaussian_filter(map_ori, sigma=3)266267 map_left = np.zeros(map.shape)268 map_left[1:,:] = map[:-1,:]269 map_right = np.zeros(map.shape)270 map_right[:-1,:] = map[1:,:]271 map_up = np.zeros(map.shape)272 map_up[:,1:] = map[:,:-1]273 map_down = np.zeros(map.shape)274 map_down[:,:-1] = map[:,1:]275276 peaks_binary = np.logical_and.reduce((map>=map_left, map>=map_right, map>=map_up, map>=map_down, map > param['thre1'])) 277 peaks = list(zip(np.nonzero(peaks_binary)[1], np.nonzero(peaks_binary)[0])) # note reverse278 peaks_with_score = [x + (map_ori[x[1],x[0]],) for x in peaks]279 id = range(peak_counter, peak_counter + len(peaks))280 peaks_with_score_and_id = [peaks_with_score[i] + (id[i],) for i in range(len(id))]281282 all_peaks.append(peaks_with_score_and_id)283 peak_counter += len(peaks)284##openpose285#openpose286 connection_all = []287 special_k = []288 mid_num = 10289290for k in range(len(mapIdx)):291 score_mid = paf_avg[:,:,[x-19 for x in mapIdx[k]]]292 candA = all_peaks[limbSeq[k][0]-1]293 candB = all_peaks[limbSeq[k][1]-1]294 nA = len(candA)295 nB = len(candB)296 indexA, indexB = limbSeq[k]297if(nA != 0 and nB != 0):298 connection_candidate = []299for i in range(nA):300for j in range(nB):301 vec = np.subtract(candB[j][:2], candA[i][:2])302#303#print('vec0:',vec[0],'vec1:',vec[1])304# #305 norm = math.sqrt(vec[0]*vec[0] + vec[1]*vec[1]+0.1)306 vec = np.divide(vec, norm)307308 startend = list(zip(np.linspace(candA[i][0], candB[j][0], num=mid_num), \309 np.linspace(candA[i][1], candB[j][1], num=mid_num)))310311 vec_x = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 0] \312for I in range(len(startend))])313 vec_y = np.array([score_mid[int(round(startend[I][1])), int(round(startend[I][0])), 1] \314for I in range(len(startend))])315316 score_midpts = np.multiply(vec_x, vec[0]) + np.multiply(vec_y, vec[1])317#318#print('norm',norm)319# #320 score_with_dist_prior = sum(score_midpts)/len(score_midpts) + min(0.5*oriImg.shape[0]/norm-1, 0)321322 criterion1 = len(np.nonzero(score_midpts > param['thre2'])[0]) > 0.8 * len(score_midpts)323 criterion2 = score_with_dist_prior > 0324if criterion1 and criterion2:325 connection_candidate.append([i, j, score_with_dist_prior, score_with_dist_prior+candA[i][2]+candB[j][2]])326327 connection_candidate = sorted(connection_candidate, key=lambda x: x[2], reverse=True)328 connection = np.zeros((0,5))329for c in range(len(connection_candidate)):330 i,j,s = connection_candidate[c][0:3]331if(i not in connection[:,3] and j not in connection[:,4]):332 connection = np.vstack([connection, [candA[i][3], candB[j][3], s, i, j]])333if(len(connection) >= min(nA, nB)):334break335336 connection_all.append(connection)337else:338 special_k.append(k)339 connection_all.append([])340##openpose341#openpose342# last number in each row is the total parts number of that person343# the second last number in each row is the score of the overall configuration344 subset = -1 * np.ones((0, 20))345 candidate = np.array([item for sublist in all_peaks for item in sublist])346347for k in range(len(mapIdx)):348if k not in special_k:349 partAs = connection_all[k][:,0]350 partBs = connection_all[k][:,1]351 indexA, indexB = np.array(limbSeq[k]) - 1352353for i in range(len(connection_all[k])): #= 1:size(temp,1)354 found = 0355 subset_idx = [-1, -1]356for j in range(len(subset)): #1:size(subset,1):357if subset[j][indexA] == partAs[i] or subset[j][indexB] == partBs[i]:358 subset_idx[found] = j359 found += 1360361if found == 1:362 j = subset_idx[0]363if(subset[j][indexB] != partBs[i]):364 subset[j][indexB] = partBs[i]365 subset[j][-1] += 1366 subset[j][-2] += candidate[partBs[i].astype(int), 2] + connection_all[k][i][2]367elif found == 2: # if found 2 and disjoint, merge them368 j1, j2 = subset_idx369print ("found = 2")370 membership = ((subset[j1]>=0).astype(int) + (subset[j2]>=0).astype(int))[:-2]371if len(np.nonzero(membership == 2)[0]) == 0: #merge372 subset[j1][:-2] += (subset[j2][:-2] + 1)373 subset[j1][-2:] += subset[j2][-2:]374 subset[j1][-2] += connection_all[k][i][2]375 subset = np.delete(subset, j2, 0)376else: # as like found == 1377 subset[j1][indexB] = partBs[i]378 subset[j1][-1] += 1379 subset[j1][-2] += candidate[partBs[i].astype(int), 2] + connection_all[k][i][2]380381# if find no partA in the subset, create a new subset382elif not found and k < 17:383 row = -1 * np.ones(20)384 row[indexA] = partAs[i]385 row[indexB] = partBs[i]386 row[-1] = 2387 row[-2] = sum(candidate[connection_all[k][i,:2].astype(int), 2]) + connection_all[k][i][2]388 subset = np.vstack([subset, row])389# delete some rows of subset which has few parts occur390 deleteIdx = [];391for i in range(len(subset)):392if subset[i][-1] < 4 or subset[i][-2]/subset[i][-1] < 0.4:393 deleteIdx.append(i)394 subset = np.delete(subset, deleteIdx, axis=0)395##openpose396 numPersons= nPersons(subset)397#print(subset2AIsubset(subset, numPersons))398 AIsubset = subset2AIsubset(subset,numPersons)399#print(all_peaks[i][numPersons][3]==[int(AIsubset[0][0])])400#all_peaks->all_peaks_1d401 all_peaks_1d=all_peaks2all_peaks_1d(all_peaks)402#print('numPersons:',numPersons)403#print('multi_keypoints:',listMultiKeypoints(all_peaks_1d, numPersons))404 keys=['image_id','keypoint_annotations']405 values=[]406 image_id=str(file)[:-4]407408 keypoint_annotations = dict(zip(listHuman(numPersons), listMultiKeypoints(all_peaks_1d, numPersons)))409 values.append(image_id)410 values.append(keypoint_annotations)411412 d = dict(zip(keys, values))413414 final_results.append(d)415print(final_results)416 with open('data.json', 'w') as f:417 json.dump(final_results, f)418419print(list_image_names)[{"image_id": "a0f6bdc065a602b7b84a67fb8d14ce403d902e0d","keypoint_annotations": {"human1": [261, 294, 1, 281, 328, 1, 0, 0, 0, 213, 295, 1, 208, 346, 1, 192, 335, 1, 245, 375, 1, 255, 432, 1, 244, 494, 1, 221, 379, 1, 219, 442, 1, 226, 491, 1, 226, 256, 1, 231, 284, 1], "human2": [313, 301, 1, 305, 337, 1, 321, 345, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 313, 359, 1, 320, 409, 1, 311, 454, 1, 0, 0, 0, 330, 409, 1, 324, 446, 1, 337, 284, 1, 327, 302, 1],"human3": [373, 304, 1, 346, 286, 1, 332, 263, 1, 0, 0, 0, 0, 0, 0, 345, 313, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 363, 386, 1, 361, 424, 1, 361, 475, 1, 365, 273, 1, 369, 297, 1],... } }... ]。
基于LabVIEW的人体表面肌电信号手势识别
㊀ＩＳＳＮ１６７２－４３０５ＣＮ１２－１３５２／Ｎ实㊀㊀验㊀㊀室㊀㊀科㊀㊀学ＬＡＢＯＲＡＴＯＲＹ㊀ＳＣＩＥＮＣＥ第２１卷㊀第３期㊀２０１８年６月Ｖｏｌ ２１㊀Ｎｏ ３㊀Ｊｕｎ ２０１８基于ＬａｂＶＩＥＷ的人体表面肌电信号手势识别田新宇，孙振词，许宝忠，刘㊀鸣（天津大学精仪学院，天津㊀３０００７２）摘㊀要：随着科技发展的不断进步，人类对工具的操控和使用要求不断提高；一系列新型传感器和控制设备伴随着电子科技产业发展不断产生，人们对于交互的需求也越来越智能化㊁数字化㊂其中，手势识别作为人与人之间重要的交互沟通方式在测控领域有着巨大的潜力㊂实现手势识别有多种途径，其中基于人体表面肌电流（ｓＥＭＧ）的测量方法以其更加快捷简单的使用体验深受消费者期待㊂探讨了一种基于ｓＥＭＧ的手势识别方法，结合ＬａｂＶＩＥＷ平台进行了简单的分类和应用，为实现可携带式的ｓＥＭＧ手势识别设备提供一种实用方案㊂关键词：手势识别；肌肉表面电信号；ＬａｂＶＩＥＷ中图分类号：ＴＰ２１２．３；ＴＰ３１９㊀㊀文献标识码：Ｂ㊀㊀ｄｏｉ：１０．３９６９／ｊ．ｉｓｓｎ．１６７２－４３０５．２０１８．０３．０２１ＲｅｃｏｇｎｉｔｉｏｎｏｆｂｏｄｙｓＥＭＧｓｉｇｎａｌｇｅｓｔｕｒｅｂａｓｅｄｏｎＬａｂＶＩＥＷＴＩＡＮＸｉｎ－ｙｕ，ＳＵＮＺｈｅｎ－ｃｉ，ＸＵＢａｏ－ｚｈｏｎｇ，ＬＩＵＭｉｎｇ（ＳｃｈｏｏｌｏｆＰｒｅｃｉｓｉｏｎＩｎｓｔｒｕｍｅｎｔａｎｄＯｐｔｏ－ｅｌｅｃｔｒｏｎｉｃｓＥｎｇｉｎｅｅｒｉｎｇ，ＴｉａｎｊｉｎＵｎｉｖｅｒｓｉｔｙ，Ｔｉａｎｊｉｎ３０００７２，Ｃｈｉｎａ）Ａｂｓｔｒａｃｔ：Ｗｉｔｈｔｈｅｃｏｎｔｉｎｕｏｕｓｄｅｖｅｌｏｐｍｅｎｔｏｆｓｃｉｅｎｃｅａｎｄｔｅｃｈｎｏｌｏｇｙ，ｈｕｍａｎｍａｎｉｐｕｌａｔｉｏｎａｎｄｕｓｅｏｆｔｏｏｌｓｉｍｐｒｏｖｅｃｏｎｔｉｎｕｏｕｓｌｙ．Ａｓｅｒｉｅｓｏｆｎｅｗｓｅｎｓｏｒｓａｎｄｃｏｎｔｒｏｌｄｅｖｉｃｅｓａｓｓｏｃｉａｔｅｄｗｉｔｈｔｈｅｄｅｖｅｌｏｐ⁃ｍｅｎｔｏｆｅｌｅｃｔｒｏｎｉｃｔｅｃｈｎｏｌｏｇｙｉｎｄｕｓｔｒｙｃｏｎｔｉｎｕｅｓｔｏｐｒｏｄｕｃｅａｎｄｐｅｏｐｌｅ ｓｄｅｍａｎｄｆｏｒｉｎｔｅｒａｃｔｉｏｎａｒｅａｌｓｏｉｎｃｒｅａｓｉｎｇｌｙｉｎｔｅｌｌｉｇｅｎｔ，ｄｉｇｉｔａｌ．Ａｍｏｎｇｔｈｅｍ，ｇｅｓｔｕｒｅｒｅｃｏｇｎｉｔｉｏｎａｓａｎｉｍｐｏｒｔａｎｔｃｏｍｍｕｎｉｃａｔｉｏｎｂｅ⁃ｔｗｅｅｎｐｅｏｐｌｅｉｎｔｈｅｆｉｅｌｄｏｆｍｅａｓｕｒｅｍｅｎｔａｎｄｃｏｎｔｒｏｌｈａｓｇｒｅａｔｐｏｔｅｎｔｉａｌ．Ｔｈｅｒｅａｒｅｍａｎｙｗａｙｓｔｏａ⁃ｃｈｉｅｖｅｇｅｓｔｕｒｅｒｅｃｏｇｎｉｔｉｏｎ．Ｍｅａｓｕｒｅｍｅｎｔｏｆｍｕｓｃｌｅｃｕｒｒｅｎｔ（ｓＥＭＧ）ｂａｓｅｄｏｎｈｕｍａｎｂｏｄｙｉｓｆｕｌｌｏｆｃｏｎｓｕｍｅｒｅｘｐｅｃｔａｔｉｏｎｓｂｅｃａｕｓｅｉｔ ｓｍｏｒｅｑｕｉｃｋａｎｄｅａｓｙｔｏｕｓｅ．Ｔｈｉｓｐａｐｅｒｅｘｐｌｏｒｅｓａｇｅｓｔｕｒｅｒｅｃｏｇｎｉ⁃ｔｉｏｎｍｅｔｈｏｄｂａｓｅｄｏｎｓＥＭＧａｎｄｃｏｍｂｉｎｅｓｔｈｅＬａｂＶＩＥＷｐｌａｔｆｏｒｍｆｏｒａｓｉｍｐｌｅｃｌａｓｓｉｆｉｃａｔｉｏｎａｎｄａｐｐｌｉ⁃ｃａｔｉｏｎ，ｉｎｏｒｄｅｒｔｏａｃｈｉｅｖｅｐｏｒｔａｂｌｅｓＥＭＧｇｅｓｔｕｒｅｒｅｃｏｇｎｉｔｉｏｎｅｑｕｉｐｍｅｎｔｔｏｐｒｏｖｉｄｅａｐｒａｃｔｉｃａｌｐｒｏ⁃ｇｒａｍ．Ｋｅｙｗｏｒｄｓ：ｇｅｓｔｕｒｅｒｅｃｏｇｎｉｔｉｏｎ；ｓＥＭＧ；ＬａｂＶＩＥＷ㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀㊀通讯作者：许宝忠（１９７３－），男，辽宁阜新人，博士，工程师，主要研究方向为光电技术与控制㊂㊀㊀肌电信号（ＥＭＧ）发源于作为中枢神经一部分的脊髓中的运动神经元，大脑发出的运动指令经中枢神经系统传导至人体相关肌肉时，肌肉处会产生连续的动作电位（ＡｃｔｉｏｎＰｏｔｅｎｔｉａｌ，ＡＰ）并发生肌肉收缩［１］㊂该电位信号经过皮下组织的传导，在皮肤表面处被电极检测到的信号为表面肌电流信号（ｓＥＭＧ）㊂ｓＥＭＧ信号尽管相对复杂，干扰较多，却对人体无伤害，方便使用㊂在进行不同的手势动作时，由于肌肉的收缩程度不同，每个动作在用力程度㊁持续时间以及作用肌肉纤维和电极之间的距离等因素也不同，会产生完全不同的叠加波形，测量这些特性得到某几个动作之间的规律，根据这些规律对ｓＥＭＧ信号进行分类，就可以识别出不同的手势，因此，基于ｓＥＭＧ的测控装置具有很高的应用价值㊂１㊀总体设计整个系统分为三个部分：前端信号采集传输，上位机识别处理，手势播放器控制㊂前端信号采集部分主要结合表面肌电流的相关电学特性设计了基于ＡＤＳ１２９４的信号采集电路，上位机采用ＬａｂＶＩＥＷ对采集的信号进行特征识别，并利用ＬａｂＶＩＥＷ设计了一个简单的音乐播放器，利用识别出来的手势信息，进行相关操控㊂图１为系统设计图㊂图１㊀系统设计图２㊀硬件设计２．１㊀电极采集２．１．１㊀位置肌电极的位置对信号的采集影响很大，因此需要了解人体前臂肌肉分布及主要功能，在位置选择过程中，要防止电极采集到与手势不相关的肌电信号，从而影响分类器对手势的识别㊂人体前臂肌肉分为前后两个群并分层排布，共计１８块肌肉，每一块肌肉对应的功能不尽相同［２］㊂由于传感器仅采用了两枚电极来采集肌电信号，所以需要进一步选出更加合适的２块肌肉㊂由于仅需要对拇指动作和其他四指的屈伸进行判断，选择对指浅屈肌进行信号采集较为符合应用场景㊂当电极放在肌腹（绿色位置）时测量效果最好［３］，如图２所示㊂图２㊀电极放在肌腹（绿色位置）时的测量效果２．１．２㊀电极电极是整个肌电控制系统与人体之间的接口，其必须满足一定的技术指标，才能可靠稳定地采集到人体肌电信号，电极极片的材质一般选用ＡｇＣｌ㊁Ａｇ㊁Ａｕ，其能够与皮肤产生良好的电化学反应，减小与皮肤之间的阻抗，提供稳定的㊁高信噪比的肌电信号［４］㊂同时，两极片之间的距离最好不要超过２０ｍｍ［５］㊂电极在皮肤表面的固定，可以选用扎带㊁双面胶或者导电凝胶，为了使用方便，符合可穿戴的应用要求，应选择扎带式固定方案㊂２．２㊀ＭＣＵ考虑到便携性和ＡＤＳ１２９４的相关参数［６］，选择了ＳＴＭ３２ｆ１０３作为采集系统的ＭＣＵ进行控制㊂ＳＴＭ３２是一款３２位的ＡＲＭ微控制器，该系列芯片是意法半导体（ＳＴ）公司出品，其内核是Ｃｏｒｔｅｘ－Ｍ３，最高可达７２ＭＨｚ工作频率㊂在系统中，ＳＰＩ工作在主从机模式，由ＳＴＭ３２发起对ＳＰＩ接口的写操作，从而配置ＡＤＳ１２９４，并通过ＳＰＩ从ＡＤＳ１２９４中读取ｓＥＭＧ数据，当ＡＤＳ１２９４完成数据采集并使／ＤＲＤＹ下降沿到来时，便会触发外部中断，从而触发在ＡＲＭ中的中断响应程序，使单片机得到ＳＰＩ中数据，随后，ＳＴＭ３２利用串口将编码后的数据发送至上位机，进行下一步的数据处理㊂２．３㊀数据上传在ＭＣＵ与上位机通讯环节，采用了基于ＨＣ－０５的蓝牙串口通讯模块㊂ＨＣ－０５嵌入式蓝牙串口通讯模块具有两种工作模式：命令响应工作模式和自动连接工作模式，在自动连接工作模式下模块又可分为主（Ｍａｓｔｅｒ）㊁从（Ｓｌａｖｅ）和回环（Ｌｏｏｐｂａｃｋ）三种工作角色㊂当模块处于自动连接工作模式时，将自动根据事先设定的方式连接数据传输；当模块处于命令响应工作模式时能执行下述所有ＡＴ命令，用户可向模块发送各种ＡＴ指令，为模块设定控制参数或发布控制命令㊂通过控制模块外部引脚输入电平，可以实现模块工作状态的动态转换㊂用ＳＴＭ３２Ｆ１０３的ＵＳＡＲＴ１串口通路，９６００的波特率作为串口配置协议㊂ＨＣ－０５蓝牙模块使用从机（Ｓｌａｖｅ）模式，并用ＡＴ指令设置串口配置协议，实现了与上位机的通讯㊂３㊀软件设计３．１㊀ＭＣＵ程序设计主程序先进行串口和相关ＩＯ的初始化，随后进入轮询，通过不断检测标志信号判断是否接收到可用的ｓＥＭＧ数据，采集到数据后再进行串口通讯的相关配置，将数据最终发送给上位机，图３为ＭＣＵ程序设计图㊂图３㊀ＭＣＵ程序设计图３．２㊀上位机软件整体设计上位机程序有四个主要部分：第一，利用ｖｉｓａ模块按照一定的频率接收肌电信号；㊀田新宇，等：基于ＬａｂＶＩＥＷ的人体表面肌电信号手势识别第二，按照窗口长度２５０ｍｓ和增量７０ｍｓ将采集到的肌电信号进行分割；第三，采用事件结构调用ＭＡＴＬＡＢ对肌电信号进行计算和识别；第四，将识别结果返回音乐播放器ｖｉ㊂图４为上位机软件整体设计㊂图４㊀上位机软件整体设计图３．３㊀ＬａｂＶＩＥＷ串口通讯上位机采用了ＬａｂＶＩＥＷ进行数据的接收和处理，在数据接收时，主要采用了ＬａｂＶＩＥＷ中的ｖｉｓａ㊂ｖｉｓａ模块首先需要进行串口设置，在本系统中，只需要设置串口号和波特率，其余采用默认，另外，在实际操作中，还使用了串口调试工具ｓｓｃｏｍ３２来查看通讯效果㊂接下来，使用ｖｉｓａｒｅａｄ从串口中读取数据，利用其中的Ｒｅａｄｂｕｆｆｅｒ进行读取操作，整个数据读取完毕后，还需用ｖｉｓａｃｌｏｓｅ结束串口读取，释放串口资源㊂３．４㊀基于重叠窗口的信号采集在信号流中，信号的长度称为窗口，一般而言，越长的窗口包含的肌电信息越丰富，识别率就越高，但延时也会更加严重，而且，肌电信号具有非线性和非平稳性，单独一个窗口肌电信号很难正确估计出手势㊂研究认为重叠窗口的方案可以很好的解决延时问题［７］，其窗口的增量小于窗口的长度，每一个新窗口都包含了上一个窗口的大部分数据，仅仅增量部分是新采集到的肌电信号㊂在采集增量部分的肌电信号时，同时完成对上一个窗口的计算处理并得出判定㊂对比不重叠窗口方案，重叠窗口方案在获取多个判定并进行平滑处理时，仅需要等待若干增量的时间即可，这样才可能满足肌电控制３００ｍｓ的延时要求［８］㊂在采集过程中，采用了ＬａｂＶＩＥＷ中的 生产者－消费者 结构，并使消费速度大于生产速度，始终保持队列的缓冲区为空㊂每当加工完成一段２５０ｍｓ的数据段，即完成一次事件并触发数据计算识别部分，进而在第三环节中完成对该段肌电信号的计算处理和识别，并实时展示识别结果㊂在消费者结构中，每２５０ｍｓ的数据按照重叠窗口方案要求会存在１８０ｍｓ的重合部分㊂图５是生产者环节和消费者环节的程序㊂图５㊀生产者环节和消费者环节的程序３．５㊀数学算法原始肌电信号并不能直接用于分类识别，因为其含有较多的无效信息㊂通过查阅资料［９］，选择了绝对均值（ＭＡＶ）㊁方差（ＶＡＲ）㊁自回归系数（ＡＲ）作为特征判别的依据［１０］，用Ｎ表示窗口内的肌电信号的长度，用ｘｉ表示数据点，部分特征的定义和计算公式如下㊂３．５．１㊀绝对均值对一个窗口内的肌电信号幅值取绝对值，然后求和并求其均值，即为该段肌电信号的绝对均值，该特征常被用于肌电信号的动作发起检测㊂其计算公式如下：ＭＡＶ＝１ＮðＮｉ＝１ｘｉ３．５．２㊀方差对于一组数据，将各个数据点与该组数据平均值的差的平方求和，然后求和的平均值，即被定义为该组数据的方差㊂其计算公式为：ＶＡＲ＝１Ｎ－１ðＮｉ＝１ｘｉ－ｘ－()３．５．３㊀自回归系数根据肌电信号的产生机制，肌电信号可以等价为零均值白噪声过程激励线性系统的输出㊂通过对３７肌电信号建立参数模型，可以进行肌电信号的分析和处理㊂ＡＲ模型的数学表达式如下：ｘｉ＝ｗｉ－ðＰｋ＝１ａｋｘｎ－ｋ在ＭＡＴＬＡＢ建立相应识别模型后，通过ＭＡＴ⁃ＬＡＢ－ＬａｂＶＩＥＷ联合编程可以实现整个识别过程，其中，ＬａｂＶＩＥＷ负责将数据进行上位机收取和重叠加窗，ＭＡＴＬＡＢ进行特征提取㊁降维并最终得到手势结果㊂３．６㊀音乐播放器控制识别结果返回ＬａｂＶＩＥＷ后，使用条件结构将不同手势分别作为 播放 ㊁ 暂停 ㊁ 上一首 ㊁ 下一首 的条件，实现了通过手势对音乐播放器的控制㊂图６为音乐播放器控制部分的程序框图㊂图６㊀音乐播放器控制部分的程序框图４㊀系统调试将固定有电极的扎带戴在手臂指浅屈肌位置上，打开前端信号采集系统后，ＳＴＭ３２开始将ｓＥＭＧ信号不断上传至上位机，经过ＬａｂＶＩＥＷ和ＭＡＴＬＡＢ的共同处理后，实现了通过手势对音乐播放器的控制，系统处理的效果和速度较好，但由于放置位置难以达到十分准确等原因，其识别精度还有待提高㊂图７为上位机操作画面，图８为实际穿戴测试图㊂图７㊀上位机操作画面㊀㊀㊀㊀图８㊀实际穿戴测试图５㊀结语肌肉电信号的测量和处理在实际生产生活中有很大的应用潜力㊂肌电信号的检测，相关数据的处理是本系统的难点㊂本文基于ＡＤＳ１２９４芯片实现了对肌电信号的实时采集，依托ＳＴＭ３２ｆ１０３控制器进行数据协调和上位机通讯，本系统还基于Ｌａｂ⁃ＶＩＥＷ实现了数据处理㊁实时识别和简单控制；另外，为实现可携带的目标，整个系统上㊁下位机通讯采用了蓝牙协议，提高了系统的便捷性㊂参考文献（Ｒｅｆｅｒｅｎｃｅｓ）：［１］㊀李强．表面肌电信号的运动单位动作电位检测［Ｄ］．合肥：中国科学技术大学，２００８．［２］㊀卢祖能．实用肌电图学［Ｍ］．北京：人民卫生出版社，２０００：５０２－５０４．［３］㊀ＬｕｃａＣ．Ｊ．Ｄ．ＰｒａｃｔｉｃｕｍｏｎｔｈｅＵｓｅｏｆＳｕｒｆａｃｅＥＭＧＳｉｇｎａｌｓｉｎＭｏｖｅ⁃ｍｅｎｔＳｃｉｅｎｃｅｓ（ｖ１．５）［ＥＢ／ＯＬ］．［２００７－０８－３０］．ｈｔｔｐ：／／ｗｗｗ．ｄｅｌｓｙｓ．ｃｏｍ／ｋｎｏｗｌｅｄｇｅＣｅｎｔｅｒ／Ｐｒａｃｔｉｃｕｍ．ｈｔｍｌ．［４］㊀ＳｍｉｔｈＬ．Ｓｔａｎｄａｒｄｓｆｏｒｓｕｒｆａｃｅｅｌｅｃｔｒｏｍｙｏｇｒａｐｈｙ：ＴｈｅＥｕｒｏｐｅａｎｐｒｏ⁃ｊｅｃｔＳｕｒｆａｃｅＥＭＧｆｏｒｎｏｎ－ｉｎｖａｓｉｖｅａｓｓｅｓｓｍｅｎｔｏｆｍｕｓｃｌｅｓ（ＳＥＮＩ⁃ＡＭ）［Ｊ］．ＣｌｉｎｉｃａｌＢｉｏｍｅｃｈａｎｉｃｓ，２０１６，２４（１）：１０８－１１２．［５］㊀ＤａｙＳ．ＩｍｐｏｒｔａｎｔｆａｃｔｏｒｓｉｎｓｕｒｆａｃｅＥＭＧｍｅａｓｕｒｅｍｅｎｔ［ＥＢ／ＯＬ］．［２００２－０１－１７］．ｈｔｔｐ：／／ｗｗｗ．ａｎｄｒｅｗｓｔｅｒｉａｎ．ｃｏｍ／ｃｏｕｒｓｅｓ／２１４／ＥＭＧ＿ｍｅａｓｕｒｅｍｅｎｔ＿ａｎｄ＿ｒｅｃｏｒｄｉｎｇ．ｐｄｆ．［６］㊀林锦荣，谭北海，谢胜利，等．基于ＡＤＳ１２９４的表面肌电信号采集系统的设计［Ｊ］．医疗卫生装备，２０１５，３６（１）：５－７．［７］㊀ＳｍｉｔｈＬ．Ｈ．，ＨａｒｇｒｏｖｅＬ．Ｊ．，ＬｏｃｋＢ．Ａ．，ｅｔａｌ．Ｄｅｔｅｒｍｉｎｉｎｇｔｈｅｏｐｔｉ⁃ｍａｌｗｉｎｄｏｗｌｅｎｇｔｈｆｏｒｐａｔｔｅｒｎｒｅｃｏｇｎｉｔｉｏｎ－ｂａｓｅｄｍｙｏｅｌｅｃｔｒｉｃｃｏｎ⁃ｔｒｏｌ：Ｂａｌａｎｃｉｎｇｔｈｅｃｏｍｐｅｔｉｎｇｅｆｆｅｃｔｓｏｆｃｌａｓｓｉｆｉｃａｔｉｏｎｅｒｒｏｒａｎｄｃｏｎ⁃ｔｒｏｌｌｅｒｄｅｌａｙ［Ｊ］．ＩＥＥＥＴｒａｎｓａｃｔｉｏｎｓｏｎＮｅｕｒａｌＳｙｓｔｅｍｓａｎｄＲｅｈａ⁃ｂｉｌｉｔａｔｉｏｎＥｎｇｉｎｅｅｒｉｎｇ，２０１１，１９（２）：１８６－１９２．［８］㊀ＥｎｇｌｅｈａｒｔＫ．，ＨｕｄｇｉｎｓＢ．Ａｒｏｂｕｓｔ，ｒｅａｌ－ｔｉｍｅｃｏｎｔｒｏｌｓｃｈｅｍｅｆｏｒｍｕｌｔｉｆｕｎｃｔｉｏｎｍｙｏｅｌｅｃｔｒｉｃｃｏｎｔｒｏｌ［Ｊ］．ＩＥＥＥＴｒａｎｓａｃｔｉｏｎｓＯｎＢｉｏ⁃ｍｅｄｉｃａｌＥｎｇｉｎｅｅｒｉｎｇ，２００３，５０（７）：８４８－８５４．［９］㊀卢海峰，江朝元，阳小光．基于串口通信的在线监测系统关键技术研究［Ｊ］．仪器仪表学报，２００６，２７（Ｚ３）：２０４３－２０４４．［１０］㊀ＣＨＥＮＹ．ＴｈｅＳｔｕｄｙｏｆＡｎｔｈｒｏｐｏｍｏｒｐｈｉｃＰｒｏｓｔｈｅｔｉｃＨａｎｄＣｏｎｔｒｏｌＢａｓｅｄｏｎＥｌｅｃｔｒｏｍｙｏｇｒａｐｈｙ（ＥＭＧ）［Ｄ］．Ｇｕａｎｇｚｈｏｕ：ＳｏｕｔｈＣｈｉｎａＵｎｉｖｅｒｓｉｔｙｏｆＴｅｃｈｎｏｌｏｇｙ，２０１４．收稿日期：２０１７－０７－２６修改日期：２０１７－０９－１１作者简介：田新宇（１９９７－），男，山东滨州人，２０１５级工程科学实验班学生，主要研究方向为测控技术与仪器㊂㊀。