TEP Story Archives: Developing a Fully Automatic 3D Video System with Human Eyes

Share Button

Dr. Xiaolin Zhang
CEO & CTO of Bi2-Vision Co., Ltd./Associate Professor at Tokyo Institute of Technology/TEP Member

A new technology that revolutionized the production of 3D video

The 3D market is becoming a major movement that encompasses the movie, television, video game, and home appliance industries. There is a researcher in Japan who has had his eye on the possibilities of 3D video for many years and is now gaining global attention thanks to his unique technology.

This researcher is Dr. Xiaolin Zhang, who is an associate professor at Tokyo Institute of Technology and the CEO & CTO of Bi2-Vision Co., Ltd (Bi2-Vision). Dr. Zhang is called "the genius who recreated the human eye with cameras," and his technology is now changing the way that people think about video production.

Despite the booming popularity of 3D video, the technology behind its production has yet to catch up. Normally when creating 3D video, two cameras are used to recreate the left and right eyes of a human being, and the shooting angles are finely adjusted while manually editing distortions in the captured video.

This process is extremely complicated, and unless the distortion is adjusted in an optimal manner, the video ends up causing eye strain. It takes an enormous amount of time and money to complete 3D content, and that is one reason why it has not fully taken off. Dr. Zhang developed the first system in the world to completely automate these processes, which transformed the production environment for 3D video.

Merging robotic engineering and medical technologies

Dr. Zhang's unique background had a great influence on how he achieved this fully automatic 3D video system. Inspired by the Japanese animation about robots called "Astro Boy," Dr. Zhang left his native China in 1987 to study abroad in Japan, which was known as a technological powerhouse. He started research into the vision control of industrial robots at the graduate school of Yokohama National University.

However, system control engineering at the time was unable to decipher the complicated movements of human eyes. That is why he transferred to Tokyo Medical and Dental University Faculty of Medicine in 1995. He conducted intensive research into the mechanisms of eye movement and retina information processing to discover how the two eyes of a human being move to generate 3D images.

From 2003, Dr. Zhang worked as an associate professor at Tokyo Institute of Technology, where he started development of a robotic eye using an active stereo camera that recreates human eye movement, based on his research into robotic engineering technology and his knowledge of biology/anatomy. The novelty of his research also caught the attention of the Japanese government, and by 2009 his technology had reached a practical level thanks to funding from the Ministry of Education, Culture, Sports, Science, and Technology.

Developing a robot that uses two eyes (cameras) to detect human faces and move accordingly

Bi2-Vision entered the 3D market in the same year as the movie "Avatar"

Dr. Zhang founded Bi2-Vision in August 2009, as a university-based venture company in order to commercialize his revolutionary technology. However, the robotics industry was still in its early days and there was not yet a mature business environment. That is why he decided to target the 3D market. He realized that he would be able to repurpose his technology and its market would be established earlier than that of robots.

In 2010, several months after Bi2-Vision was founded, attention quickly turned to 3D video thanks to the global blockbuster movie called "Avatar." The world's video production sites were seeking a product that would enable low-eye strain 3D video to be produced quickly and at a low cost.

In 2011, video production giant IMAGICA Corp. adopted Dr. Zhang's 3D video system, and now Bi2-Vision is starting to actively approach customers both in Japan and overseas. Dr. Zhang now plans to utilize his technology and the experience gained in Japan to focus on the Chinese 3D video market, which is thought to be two years behind. The Bi2-Vision product is currently being adjusted to match the market needs of China.

A Japanese president and Chinese engineer working in tandem

"I am a beginner when it comes to running a company, and I am really more of a researcher," says Dr. Zhang, who welcomed Mr. Ryuichi Murakami as the president of Bi2-Vision in 2010. Mr. Murakami retired as the president of a company affiliated with IBM and is now an angel member of TX Entrepreneur Partners. Impressed by the technology and personality of Dr. Zhang, he decided to not only provide financial assistance as an investor, but also assist Bi2-Vision with the knowledge on company management that he built up during his experience as president.

The employees of Bi2-Vision are all university researchers or new Ph.D.s and those with experience in the corporate world are multinational staff from China, South Korea, and Japan. In regard to the employees with almost zero real-world experience, "I had to teach them everything from scratch, including how to clean the office, how to conduct sales, and how to develop sales channels," recalls Mr. Murakami with a laugh, his eyes full of life. "I want to build a world where ventures born from leading Japanese technology lead the business world when my grandchildren become adults," he says with conviction.

Leaving management of the company up to the trustworthy Mr. Murakami enables Dr. Zhang to focus on his own research. Dr. Zhang currently instructs about 20 students at university while developing new technology on a daily basis. Of course, his eye is still on the goal of robot development. "Dr. Zhang's mind is a treasure trove of ideas. My job is simply to bring out those ideas at the right time," says Mr. Murakami, showing his confidence in the future potential of the company.

Although the company is headed by a Japanese president and a Chinese engineer, Dr. Zhang says that "We are both Asians." His dream is to create a global high-tech venture from Asia. "Disney World shows the development history of human civilization, but there is no mention of Asia. I want to show the world what Asians can do," says Dr. Zhang, who certainly captures the spirit of venture entrepreneurialism.

Share Button