DLR Institute of Robotics and Mechatronics: Dr. Heiko Hirschmüller

Heiko Hirschmüller
Ph.D., M.Sc., Dipl.-Inform. (FH)

German Aerospace Center (DLR)
Institute of Robotics and Mechatronics
Department of Perception and Cognition
Münchner Str. 20
D-82234 Wessling
Germany

Research Interests

Stereo matching
Stereo visual odometry and image registration
Stereo vision based 3D reconstruction

Semi-Global Matching (SGM)

SGM is a stereo matching method that is based on pixel-wise matching, supported by a global smoothness function that is optimized along multiple paths (Hirschmüller, 2008, 2006 and 2005). It has a regular algorithmic structure and uses simple operations (i.e. in the core loop just adding and comparing integer values) and is therefore well suited for parallel implementation on the CPU using vector commands as well as on the GPU (Ernst and Hirschmüller, 2008) and FPGA (Hirschmüller, 2011). Census is used as matching cost for radiometric robustness (Hirschmüller and Scharstein 2009). The method is quite insensitive to the choice of parameters, which means that it usually does not require parameter tuning.

	SGM is used for processing aerial and satellite images from pushbroom and full frame cameras (Hirschmüller, 2011; Hirschmüller and Bucher, 2010; Börner et al., 2008; Hirschmüller et al., 2005), resulting in high quality 2.5D surface models in the ground sampling resolution of the input images.
	SGM is used for 3D reconstruction of buildings from UAV images (Schmid et al., 2011).
	SGM is used for mobile robot navigation (Chilian et al., 2011; Görner et al., 2010). A GPU implementation of SGM is currently used (Ernst and Hirschmüller, 2008).
	SGM is part of the 6d-Vision system that has been developed by Daimler researchers for driver assistance applications. SGM is implemented on a FPGA for real-time processing. Techniques for stereo matching with slightly mis-calibrated cameras have been developed in this context (Hirschmüller and Gehrig, 2009). (The image on the left is a courtesy of Stefan Gehrig, Daimler AG).

Awards

Carl Pulfrich Award 2011 for outstanding achievements in Photogrammetry, Remote Sensing and Earth Imaging applications, Photogrammetric Week, 5th September 2011, Stuttgart, Germany. The corresponding paper explains SGM and its applications.

German Aerospace Center (internal) Science Award 2009 (DLR-Wissenschaftspreis 2009) for "Stereo Processing by Semi-Global Matching and Mutual Information" on 26th November 2009 in Berlin. See Video of award ceremony (German, forward to 1:30 to start).

Journal Papers

Heiko Hirschmüller and Daniel Scharstein (2009), Evaluation of Stereo Matching Costs on Images with Radiometric Differences, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 31(9), September 2009, pp. 1582-1599.

Heiko Hirschmüller (2008), Stereo Processing by Semi-Global Matching and Mutual Information, in IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 30(2), February 2008, pp. 328-341.

Heiko Hirschmüller, Peter R. Innocent and Jon Garibaldi (2002), Real-Time Correlation-Based Stereo Vision with Reduced Border Errors, International Journal of Computer Vision, Volume 47 (1/2/3), April-June 2002, pp. 229-246.

Peer-reviewed Conference Papers

Annett Chilian, Heiko Hirschmüller and Martin Görner, Multisensor Data Fusion for Robust Pose Estimation of a Six-Legged Walking Robot, IEEE International Conference on Intelligent Robots and Systems (IROS) in September 2011 in San Francisco, CA, USA.

Korbinian Schmid, Heiko Hirschmüller, Andreas Döml, Iris Grixa, Michael Suppa and Gerd Hirzinger, View planning for multi-view stereo 3D reconstruction using an autonomous multicopter, International Conference on Unmanned Aircraft Systems in May 2011 in Denver, CO, USA (also appeared in the Journal of Intelligent and Robotic Systems, 2012, Volume 65, pp. 309-323).

Florian-Michael Adolf and Heiko Hirschmüller (2010), Surface Meshing and Simplification of High Resolution Urban Terrain Data for 3D Path Planning, UAV in June 2010 in Dubai.

Annett Chilian and Heiko Hirschmüller (2009), Stereo Camera Based Navigation of Mobile Robots on Rough Terrain, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) in October 2009 in St. Louis, MO, USA.

Heiko Hirschmüller and Stefan Gehrig (2009), Stereo Matching in the Presence of Sub-Pixel Calibration Errors, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 20-25 June 2009, Miami, Florida, USA, pp. 437-444.

Ines Ernst and Heiko Hirschmüller (2008), Mutual Information based Semi-Global Stereo Matching on the GPU, in Proceedings of the International Symposium on Visual Computing (ISVC08), 1-3 December 2008, Las Vegas, Nevada, USA.

Anko Börner, Heiko Hirschmüller, Karsten Scheibe, Michael Suppa and Jürgen Wohlfeil (2008), MFC - A Modular Line Camera for 3D World Modelling, Robot Vision: Second International Workshop RobVis 2008, LNCS 4931, 18.-20. February, Auckland, New Zealand, pp. 319-326.

Heiko Hirschmüller and Daniel Scharstein (2007), Evaluation of Cost Functions for Stereo Matching, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 18-23 June 2007, Minneapolis, Minnesota, USA.

Christian Ott, O. Eiberger, W. Friedl, B. Bauml, U. Hillenbrand, C. Borst, A. Albu-Schaffer, B. Brunner, H. Hirschmüller, S. Kielhofer, R. Konietschke, M. Suppa, T. Wimbock, F. Zacharias, G. Hirzinger (2006), A Humanoid Two-Arm System for Dexterous Manipulation, 6th IEEE-RAS International Conference on Humanoid Robots, December 2006, Genoa, Italy, pp. 276-283.

Heiko Hirschmüller (2006), Stereo Vision in Structured Environments by Consistent Semi-Global Matching, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 17-22 June 2006, New York, NY, USA, pp. 2386-2393.

Heiko Hirschmüller, Frank Scholten, Gerd Hirzinger (2005), Stereo Vision Based Reconstruction of Huge Urban Areas from an Airborne Pushbroom Camera (HRSC), in Lecture Notes in Computer Science: Pattern Recognition, Proceedings of the 27th DAGM Symposium, 30 August - 2 September 2005, Vienna, Austria, Volume 3663, pp. 58-66.

Heiko Hirschmüller (2005), Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 20-26 June 2005, San Diego, CA, USA, Volume 2, pp. 807-814.

Heiko Hirschmüller, Peter R. Innocent and Jon M. Garibaldi (2002), Fast, Unconstrained Camera Motion Estimation from Stereo without Tracking and Robust Statistics, in Proceedings of the 7th International Conference on Control, Automation, Robotics and Vision, 2-5 December 2002, Singapore, pp. 1099-1104.

Heiko Hirschmüller (2001), Improvements in Real-Time Correlation-Based Stereo Vision, in Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision, 9-10 December 2001, Kauai, Hawaii, pp. 141-148.

Ph.D. Thesis

Heiko Hirschmüller (2003), Stereo Vision Based Mapping and Immediate Virtual Walkthroughs, Ph.D. Thesis, De Montfort University, Leicester, UK, June 2003.

Other Conference Papers

Heiko Hirschmüller, Semi-Global Matching - Motivation, Developments and Applications, Invited Paper at the Photogrammetric Week, September 2011 in Stuttgart, Germany, pp. 173-184.

Martin Görner, Annett Chilian and Heiko Hirschmüller (2010), Towards an Autonomous Walking Robot for Planetary Surfaces, in Proceedings of the 10th International Symposium on Artificial Intelligence, robotics and Automation in Space (i-SAIRAS), August/September 2010 in Sapporo, Japan.

Bernhard Rebele, Armin Wedler, Maximilian Apfelbeck, Heiko Hirschmüller, Sebastian Kuss, Andreas Gibbesch, Bernd Schäfer and Gerd Hirzinger (2010), Advanced Testbed and Simulation Environment for Planetary Exploration and Mobility Investigations, in Proceedings of the 10th International Symposium on Artificial Intelligence, robotics and Automation in Space (i-SAIRAS), August/September 2010 in Sapporo, Japan. (best poster award)

Heiko Hirschmüller and Tilman Bucher (2010), Evaluation of Digital Surface Models by Semi-Global Matching, DGPF, July 2010 in Vienna, Austria.

Anko Börner, Heiko Hirschmüller, Michael Neidhardt, Michael Nitz, Karsten Scheibe, Jürgen Wohlfeil and Sergej Zuev (2007), Ableitung photogrammetrischer Produkte mit der MFC-Kamera, Dreiländertagung 2007, Gemeinsame Jahrestagung der SGPBF, DGPF und OVG, June 2007, Muttenz, Switzerland.

Anko Börner, Michael Suppa, Thomas Abmayr, Tim Bodenmüller, Andreas Eckardt, Heiko Hirschmüller, Karsten Scheibe and Bernhard Strackenbrock (2006), Ein modulares Sensorsystem zur multiskalaren Erfassung von dreidimensionalen Objekten, Geoinformatik und Erdbeobachtung, 26. Jahrestagung der DGPF, September 2006, Berlin-Adlershof, Germany, pp. 319-326.

Heiko Hirschmüller, Helmut Mayer, G. Neukum and the HRSC-CoI Team (2006), Stereo Processing of HRSC Mars Express Images by Semi-Global Matching, International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVI, Part 4, 27-30 September 2006, Goa, India.

C. Heipke, J. Oberst, J. Albertz, M. Attwenger, P. Dorninger, E. Dorrer, M. Ewe, S. Gehrke, K. Gwinner, H. Hirschmüller, J. R. Kim, R. L. Kirk, H. Mayer, J.-P. Muller, R. Rengarajan, M. Rentsch, R. Schmidt, F. Scholten, J. Shan, M. Spiegel, M. Wählisch, G. Neukum and the HRSC-CoI Team (2006), The HRSC DTM Test, International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XXXVI, Part 4, 27-30 September 2006, Goa, India.

Johann Heindl, Martin Otter, Heiko Hirschmüller, Mirko Frommberger, Florian Siegert, Harald Heinrich (2006), The Robocoaster Simulation Platform - Path and Video Generation for an Authentic Mars Flight Simulation, 37th International Symposium on Robotics, 15-17 May 2006 in Munich, Germany.

Johann Heindl, Martin Otter, Heiko Hirschmüller, Mirko Frommberger, Norbert Sporer, Florian Siegert, Harald Heinrich (2005), The Robocoaster as Simulation Platform - Experiences from the First Authentic Mars Flight Simulation, in Proceedings of the Motion Simulator Conference, 21 September 2005, Braunschweig, Germany.

Gerd Hirzinger, Tim Bodenmüller, Heiko Hirschmüller, Rui Liu, Wolfgang Sepp, Michael Suppa, Thomas Abmayr, Bernhard Strackenbrock (2005), Photo-Realistic 3D-Modelling - From Robotics Perception Towards Cultural Heritage, in International Workshop on Recording, Modelling and Visualization of Cultural Heritage, 22-27 May 2005, Ascona, Switzerland.

Heiko Hirschmüller (2003), Real-Time Map Building from a Stereo Camera under Unconstrained 3D Motion, in Proceedings of the Faculty Research Conference, 14-15 April 2003, Leicester, UK, pp. 1-6, Faculty of Computing Sciences and Engineering, De Montfort University.

Micheal Callaghan and Heiko Hirschmüller (1998), 3-D Visualisation of Design Patterns and Java Programs in Computer Science Education, in Proceedings of the 6th Conference on Teaching of Computing, August 1998, Dublin, Ireland, pp. 37-40.

Heiko Hirschmüller, Michael Callaghan and Alan O'Callaghan (1998), Virtual Learning Environment for Object-Oriented Design Patterns, in Proceedings of Euromedia 98, 5-6 January 1998, Leicester, UK, pp. 238-245.

Other Publications

Heiko Hirschmüller (1993), Pi auf eine Million Stellen, Amiga Magazin (Sonderheft), Markt & Technik Verlag, Germany, Volume 1, pp. 64-66.

Patents

Inventors: Heiko Hirschmüller, Ines Ernst, Maximilian Buder, Anko Börner and Jürgen Wohlfeil (2009), Method and Apparatus for Reducing the Memory Requirement for Determining Disparity Values for at Least Two Stereoscopically Recorded Images, WO/2010/102840, PCT/EP2010/001598 (DE 10 2009 012 441), 12th March 2009.