Journal of Bioinformatics and Biological Engineering   
Journal of Bioinformatics and Biological Engineering(JBBE)
Frequency: Annually
Real Time Decoding for Brain-Machine Interface Applications
Full Paper(PDF, 2598KB)
There is substantial evidence that it is possible to predict movement intentions from single cell recordings in monkeys and since more recently, in humans. Such predictions, using decoding algorithms, have a large potential for clinical applications in order to drive robotic devices to be used by paralyzed patients or amputees. In spite of these advances, it is still not clear how accurate and practical the movements obtained from real neuronal devices could be. In this work, an original decoding method to perform movements to different locations was proposed and studied in realistic simulations. The method provides a high level control command to a Brain-Machine Interface device, which is a precise estimation of the target location as a function of the number of recorded neurons. Finally the method was applied to a 7 Degrees of Freedom (DOF) anthropomorphic robotic arm for reaching and grasping an object.
Keywords:Brain-Machine Interface; Neuroprosthetic Devices; Neural Decoding; Parietal Cortex
Author: Jonathan Becedas1, Rodrigo Quian Quiroga1
1.Centre for Systems Neuroscience, University of Leicester, Leicester, United Kingdom
  1. R. A. Andersen, S. Musallam, and B. Pesaran, “Cognitive neural prosthetics,” TRENDS in Cognitive Sciences, vol. 8, iss. 11, pp. 486–493, 2004.
  2. M. A. Nicolelis, “Action for thoughts,” Nature, vol. 409, pp. 403–407, 2001.
  3. J. Becedas, “Brain-machine interfaces: Basis and advances,” IEEE Transactions on Systems, Man, and Cybernetics, Part C, vol. 42, iss. 6, pp. 825–836, 2012.
  4. J. P. Donoghue, “Connecting cortex to machines: recent advances in brain interfaces,” Nature Neuroscience, vol. 5. pp. 1085–1088, 2002.
  5. M. A. Lebedev and M. A. Nicolelis, “Brain-machine interfaces: past, present and future,” Trends in Neuroscience, vol. 29, iss. 9, pp. 536–546, 2006.
  6. G. E. Loeb, “Cochlear prosthetics,” Annual Review on Neuroscience, vol. 13, pp. 357–371, 1990.
  7. R. M. Rothschild, “Neuroengineering tools/applications for bidirectional interfaces, brain-computer interfaces, and neuroprosthetic implants - A review of recent progress,” Frontiers in Neuroengineering, vol. 3, iss. 112, pp. 1–5, 2010.
  8. M. D. Serruya, N. G. Hatsopoulos, L. Paninski, M. R. Fellows, and J. P. Donoghue, “Instant neural control of a movement signal,” Nature, vol. 416, pp. 141–142, 2002.
  9. D. M. Taylor, S. I. H. Tillery, and A. B. Schwartz, “Direct cortical control of 3d neuroprosthetic devices,” Science, vol. 296, pp. 1829–1832, 2002.
  10. L. R. Hochberg, M. D. Serruya, G. M. Friehs, J. A. Mukand, M. Saleh, A. H. Caplan, A. Branner, Da. Chen, R. D. Penn, and J. P. Donoghue, “Neuronal ensemble control of prosthetic devices by a human with tetraplegia,” Nature, vol. 442, pp. 164–171, 2006.
  11. J. K. Chapin, K. A. Moxon, R. S. Markowitz, and M. A. Nicolelis, “Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex,” Nature Neuroscience, vol. 2, pp. 664–670, 1999.
  12. A. B. Schwartz, X. T. Cui, D. J. Weber, and D. W. Moran, “Brain-controlled interfaces: movement restoration with neural prosthetics,” Neuron, vol. 52, pp. 205–220, 2006.
  13. J. Wessberg, C. R. Stambaugh, J. D. Kralik, P. D. Beck, M. Laubach, J. K. Chapin, J. Kim, S. J. Biggs, M. A. Srinivasan, and M. A. Nicolelis, “Real-time prediction of hand trajectory by ensembles of cortical neurons in primates,” Nature, vol. 408, pp. 361–365, 2000.
  14. J. M. Carmena, M. A. Lebedev, R. E. Crist, J. E. O’Doherty, D. M. Santucci, D. F. Dimitrov, P. G. Patil, C. S. Henriquez, and M. A. L. Nicolelis, “Learning to control a brain-machine interface for reaching and grasping by primates,” PLoS Biology, vol. 1, iss. 2, pp. 193–208, 2003.
  15. I. Obeid and P. D. Wolf, “Evaluation of spike-detection algorithms for a brain-machine interface application,” IEEE Transactions on Biomedical Engineering, vol. 51, iss. 6, pp. 905–911, 2004.
  16. J. d. R. Millan, F. Renkens, J. Mourino, and W. Gerstner, “Non-invasive brain-actuated control of a mobile robot by human eeg,” IEEE Transactions on Biomedical Engineering, vol. 51, iss. 6, pp. 1026–1033, 2004.
  17. H. K. Kim, S. J. Biggs, D. W. Schloerb, J. M. Carmena, M. A. Lebedev, M. A. Nicolelis, and M. A. Srinivasan, “Continuous shared control for stabilizing reaching and grasping with brain-machine interfaces.,” IEEE Transactions on Biomedical Engineering, vol. 53, iss. 6, pp. 1164–1173, 2006.
  18. F. Galan, M. Nutting, E. Lew, P. W. Ferrez, G. Vanacker, J. Philips, and J. d. R. Millán, “A brain-actuated wheelchair: Asynchronous and non-invasive brain-computer interfaces for continuous control of robots,” Clinical Neurophysiology, vol. 119, pp. 2159-2169, 2008.
  19. M. Velliste, S. Perel, M. C. Spalding, A. S. Whitford, and A. B. Schwartz, “Cortical control of a prosthetic arm for self-feeding,” Nature, vol. 453, pp. 1098–1101, 2008.
  20. N. P. Birbaumer, N. Ghanayim, T. Hinterberger, I. H. Iversen, B. Kotchoubey, A. Kbler, J. Perelmouter, E. Taub, and H. Flor, “A spelling device for the paralysed,” Nature, vol. 398, pp. 297–298, 1999.
  21. J. R. Wolpaw, N. P. Birbaumer, D. J. McFarland, G. Pfurtscheller, and T. M. Vaughan, “Brain-computer interfaces for communication and control,” Clinical Neurophysiology, vol. 113, pp. 767–791, 2002.
  22. J. R. Wolpaw, “Brain-computer interfaces (bcis) for communication and control: a mini-review,” Supplements to Clinical Neurophysiology, vol. 57, pp. 607–613, 2004.
  23. B. Obermaier, G. R. Mller, and G. Pfurtscheller, “Virtual keyboard controlled by spontaneous eeg activity,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 11, pp. 422–426, 2003.
  24. T. Hinterberger, R. Veit, B. Wilhelm, N. Weiskopf, J. -J. Vatine, and N. Birbaumer, “Neuronal mechanisms underlying control of a brain-computer interface,” European Journal of Neuroscience, vol. 21, pp. 3169–3181, 2005.
  25. N. Birbaumer, “Brain-computer-interface research: coming of age,” Clinical Neurophysiology, vol. 117, pp. 479–483, 2006.
  26. M. A. Nicolelis and M. A. Lebedev, “Principles of neural ensemble physiology underlying the operation of brain-machine interfaces,” Nature Reviews Neuroscience, vol. 10, pp. 530–540, 2009.
  27. M. A. Lebedev and M. A. Nicolelis, “Brain-machine interfaces: past, present and future,” Trends in Neuroscience, vol. 29, pp. 536–546, 2006.
  28. J. d. R. Millan and J. M. Carmena, “Invasive or noninvasive: Understanting brain-machine interface technology,” IEEE Engineering in Medicine and Biology Magazine, vol. 29, iss. 1, pp. 16–22, 2010.
  29. F. E. M. -Ivaldi and L. E. Miller, “Brain machine interfaces: Computational shorter demands and clinical needs meet basic neuroscience,” Trends in Neuroscience, vol. 26, iss. 6, pp. 329–334, 2003.
  30. E. M. Schmidt, “Single neuron recording from motor cortex as a possible source of signals for control of external devices,” Annals of Biomedical Engineering, vol. 8, pp. 329–349, 1980.
  31. S. Shoham, L. M. Paninski, M. R. Fellows, N. G. Hatsopoulos, J. P. Donoghue, and R. A. Normann, “Statistical encoding model for a primary motor cortical brain-machine interface,” IEEE Transactions on Biomedical Engineering, vol. 52, iss. 7, pp. 1312–1322, 2005.
  32. J. Zhuang, W. Truccolo, C. V. –Irwin, and J. P. Donoghue, “Decoding 3-d reach and grasp kinematics from high-frequency local field potentials in primate primary motor cortex,” IEEE Transactions on Biomedical Engineering, vol. 57, iss. 7, pp. 1774–1784, 2010.
  33. B. Pesaran, J. S. Pezaris, M. Sahani, P. P. Mitra, and R. A. Andersen, “Temporal structure in neuronal activity during working memory in macaque parietal cortex,” Nature Neuroscience, vol. 5, iss. 8, pp. 805–811, 2002.
  34. S. Musallam, B. D. Corneil, B. Greger, H. Scherberger, and R. A. Andersen, “Cognitive control signals for neural prosthetics,” Science, vol. 305, pp. 258–262, 2004.
  35. R. C. Miall and D. M.Wolpert, “Forward models for physiological motor control,” Neural Networks, vol. 9, iss. 8, pp. 1265–1279, 1996.
  36. C. Kemere, G. Santhanam, B. M. Yu, A. Afshar, S. I. Ryu, T. H. Meng, and K. V. Shenoy, “Model-based decoding of reaching movements for prosthetic systems,” In Proceedings of the 26th Annual International Conference of the IEEE EMBS, pp. 4524–4528, San Francisco, Sept. 2004.
  37. A. B. Schwartz, R. E. Kettner, and A. P. Georgopoulos, “Primate motor cortex and free arm movements to visual targets in three-dimensional space. i. relations between single cell discharge and direction of movement,” The Journal of Neuroscience, vol. 8, iss. 8, pp. 2913–2927, 1988.
  38. A. P. Georgopoulos, R. E. Kettner, and A. B. Schwartz, “Primate motor cortex and free arm movements to visual targets in three-dimensional space. ii. coding of the direction of movement by a neural population,” The Journal of Neuroscience, vol. 8, iss. 8, pp. 2928–2937, 1988.
  39. R. E. Kettner, A. B. Schwartz, and A. P. Georgopoulos, “Primate motor cortex and free arm movements to visual targets in three-dimensional space. iii. position gradients and population coding of movement direction from various movement origins,” The Journal of Neuroscience, vol. 8, iss. 8, pp. 2938–2947, 1988.
  40. R. Heliot, K. Ganguly, J. Jimenez, and J. M. Carmena, “Learning in closed-loop brain machine interfaces: Modeling and experimental validation,” IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, vol. 40, iss. 5, pp. 1387–1397, 2010.
  41. R. A. Andersen, E. J. Hwang, and G. H. Mulliken, “Cognitive neural prosthetics,” Annual Review of Physiology, vol. 61, pp. 169–190, 2010.
  42. R. A. Andersen, S. Musallam, and B. Pesaran, “Selecting the signals for a brain-machine interface,” Current Opinion in Neurobiology, vol. 14, pp. 720–726, 2004.
  43. R. Q. Quiroga, L. H. Snyder, A. P. Batista, H. Cui, and R. A. Andersen, “Movement intention is better predicted than attention in the posterior parietal cortex,” The Journal of Neuroscience, vol. 26, iss. 13, pp. 3615–3620, 2006.
  44. J. Becedas and R. Q. Quiroga, “Implementation of a real time decoder for real neuroprosthetic applications,” In Proceedings of The IEEE Fifth International Conference on Bio-Inspired Computing: Theories and Applications, pp. 1337–1345, Liverpool, United Kingdom, 2010.
  45. R. A. Andersen and C. A. Buneo, “Intentional maps in posterior parietal cortex,” Annual Review on Neuroscience, vol. 25, pp. 189–220, 2002.
  46. R. Q. Quiroga and S. Panzeri, “Extracting information from neuronal populations: information theory and decoding approaches,” Nature Reviews Neuroscience, vol. 10, pp. 173–185, 2009.
  47. J. D. Connolly, R. A. Andersen, and M. A. Goodale, “FMRi evidence for a parietal reach region in the human brain,” Experimental Brain Research, vol. 153, pp. 140–145, 2003.
  48. E. Pereira, J. Becedas, I. Payo, F. Ramos, and V. Feliu, “Control of flexible manipulators. Theory and practice,” Robot Manipulators Trends and Development, INTECH, pp. 267–296, 2010.
  49. J. Becedas, J. R. Trapero, V. Feliu, and H. S. –Ramirez, “Adaptive controller for single-link flexible manipulators based on algebraic identification and generalized proportional integral control,” IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics, vol. 39, iss. 3, pp. 533–545, 2009.
  50. R. C. Gonzalez and R. E. Wood, “Digital Image Processing,” Prentice Hall, 2002.
  51. F. Rieke, D. Warland, R. d. R. v. Stevenink, and W. Bialek, Spikes: exploring the neural code, MIT press paperback edition, 1999.
  52. A. Pouget and T.e Sejnowski, “Spatial transformations in the parietal cortex using basis functions,” Journal of Cognitive Neuroscience, vol. 9, iss. 2, pp. 222–237, 1997.
  53. S. Ben Hamed, J. –R. Duhamel, F. Bremmer, and W. Graf, “Visual receptive field modulation in the lateral intraparietal area during attentive fixation and free gaze,” Cerebral Cortex, vol. 12, iss. 3, pp. 234–245, 2002.
  54. R. A. Andersen, G. K. Essick, and R. M. Siegel, “The encoding of spatial location by posterior parietal neurons,” Science, vol. 230, iss. 4724, pp. 456–458, 1985.
  55. L. H. Snyder, A. P. Batista, and R. A. Andersen, “Coding of intention in the posterior parietal cortex,” Nature, vol. 386, pp. 167–170, 1997.
  56. C. A. Buneo, M. R. Jarvis, A. P. Batista, and R. A. Andersen, “Direct visuomotor transformations for reaching,” Nature, vol. 416, pp. 632–636, 2002.
  57. L. J. Toth and J. A. Assad, “Dynamic coding of behaviourally relevant stimuli in parietal cortex,” Nature, vol. 415, pp. 165–168, 2002.
  58. M. Hauschild, I. H. Mulliken, Fineman, G. E. Loeb, and R. A. Andersen, “Cognitive signals for brain-machine interfaces in posterior parietal cortex include continuous 3d trajectory commands,” Proceedings of the natural academy of sciences, vol. 109, iss. 42, pp. 17075–17080, 2012.