Home | Publications | Members | Contact | Projects | Publicity | Equipment

Augmented Reality User Interface for Atomic Force Microscopes (AFM)

To develop a human-machine interface for atomic force microscope (AFM) based nano-scale manipulation. A haptic device lets the user control the position of the AFM-probe and relays measured forces to his fingertip. The user sees the topography of the nano-surface, including surface interactions, and probe positions in a realtime computer graphics environment.

An Omega 3-DOF haptic device is interfaced with an M5 Atomic Force Microscope through a control PC at high bandwidth. As imaging from the AFM is non-realtime, a virtual simulation is employed to estimate the nano-world, including modelling nano-scale deformations. The nano-surface is therefore represented by a spline, and the probe as a sphere. An efficient collision detection algorithm determines whenever the probe penetrates the surface and gives the geometry of contact.

Forces are determined from the cantilever deflections, measured from the AFM. As forces are coupled in the x and z directions from the direct AFM data, force modelling and the virtual environment are used to decouple and determine forces as a 3D vector.

Control strategies will be implemented to ensure stability of manipulation, and will be implemented for task-based guided nanomanipulation.

Modelling the interaction of the probe and the surface. The interaction of the afm-probe and the surface is modelled in realtime 3D. The surface is therefore represented by a spline and the probe as a sphere. An efficient collision detection algorithm determines whenever the probe penetrates the surface and gives the geometry of the contact. Noncontact force models, continuum mechanics contact models (JKR and Maugis' JKR-DMT Transition) together with a simple rectangular beam model of the probe yield the interaction forces and deformations.

With a 3D computer simulation coupled with realtime force feedback, an AFM can become a nanomanipulation tool where a user can interact with nano-size entities as easily as if they were lagre objects on the desk in front of them. This expands the utility of the AFM from simply a scanning device to a manufactuing tool with which one can assemble structures that are virtually impossible to build presently. Furthermore this system can allow novice users with little training to utilize the advanced capabilities of the system and become intuitively familiar with nano-scale physics.

Video 1: Low quality [1.9 MB]
Video 1: High quality [7.2 MB]

No active members

Former Members
Onur Ozcan, Cagdas Onal, Chytra Pawashe, Wolfgang Vogl, Bernice Ma, Alkas Baybas, Stuart Anderson, Martin Rosenberg


  • C. D. Onal, and M. Sitti, "A Scaled Bilateral Control System for Experimental One-Dimensional Teleoperated Nanomanipulation," International Journal of Robotics Research, Vol. 28, No, 4, pp. 484-497, 2009. pdf
  • Onal, C., Pawashe, C., Sitti, M., "A Scaled Bilateral Control System for Experimental 1-D Teleoperated Nanomanipulation Applications," IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, pp. 483-488, October 2007.
  • W. Vogl, B. Ma, and M. Sitti, ''Augmented Reality User Interface for an Atomic Force Microscope based Nanorobotic System,'' IEEE Trans. on Nanotechnology, vol. 5, no. 4, pp. 397-406, 2006. pdf
  • W. Vogl, M. F. Zah, and M. Sitti, ''Nanomanipulation with 3D Visual and Force Feedback using Atomic Force Microscopes,'' Proc. of the IEEE Nanotechnology Conference, Munich, Aug. 2004. (Best Paper Nomination)
  • M. Sitti and H. Hashimoto, ''Teleoperated touch feedback of surfaces at the nanoscale: Modeling and experiments,'' IEEE/ASME Trans. on Mechatronics, vol. 8, no. 2, pp. 287-298, June 2003.
  • S. Horiguchi, M. Sitti, and H. Hashimoto, ''Investigation of virtual reality interface for AFM-based nanomanipulation,'' IEEJ Trans. on Electronics, Information and Systems (C), pp. 1948-1956, Dec. 2000 (in Japanese).
  • M. Sitti and H. Hashimoto, ''Tele-nanorobotics using atomic force microscope as a robot and sensor,'' Advanced Robotics, vol. 13, no. 4, pp. 417-436, 1999.

  • Funded by NSF (IIS-0448042)

    [3-D simulation of a surface
    with probe tip (sphere)]
    Click to see full screenshot
    [Hardware setup]
    Click to enlarge