The Teomirn virtual piano teacher app created by Ayato Fujii is one of the first apps I’ve seen that takes perfect advantage of the AR capabilities of the Microsoft Hololens.
RealityDome uploaded a great video of how the app could work, once the technical and finance details has been ironed out. No additional details has been released on how much content has been assembled nor what other activities are to be included in the app, though there’s been some mention of the technology behind the app in the RealityDome interview with Ayato Fujii:
“First, we capture the finger movement using a kind of multiple IR sensors (Leap Motion). The body part is captured as “Point cloud” using the Kinect Sensor.
After this, we create visualization and author using Unity. For currently viewing the device we use Microsoft Hololens.”
As with most bright ideas, the app came out of necessity. Fujii-san has been practicing piano for 10 years, and has realized that there is an easier way to learn. His training as a computer graphics designer gives him good grounding in technical solutions, and according to him, he has gathered many talented people to fill in the gaps of the app.
Original article and video interview from RealityDome.