Back in highschool, I was in the middle of basketball practice, when I suddenly felt an acute pain in my knee. I had no idea what had happened to me. After visits to several different clinics, none of which could identify the problem, I finally found an orthopedist who accurately guessed the cause of my pain: The anterior cruciate ligament — one of four major ligaments of the knee, which controls anterior movements of the tibia — had been severed.
Before performing an endoscopy to check my ligament’s condition, the doctor drew a simple diagram of the knee to explain where the ligament is and how it could’ve been torn apart. Having never heard of the anterior cruciate ligament, it was difficult back then to visualize the problem from a rudimentary drawing, or know exactly where and what the ligament was.
Yesterday, however, I picked up my tablet, and with just a few swipes of my fingers I was able to pull up a 3-D illustration of the internal knee and highlight the ligament, which is hidden within it. Not only that but I could rotate the image to see how the fibrous tissue looks from all angles, and how it aligns with the bones, muscles, nerves and blood vessels.
I was using teamLabBody, an app that just last month won Best VizSim Project at the Unity Awards, hosted by the San Francisco-based Unity Technologies, which markets the game engine of the same name. The VizSim category commends applications built using Unity for “real-world grounded projects for visualization and simulation.” This year was the first time anyone from Japan had won a Unity award.
TeamLabBody, the world’s first human-anatomy app to offer detailed 3-D images, is the brainchild of Kazuomi Sugamoto, an orthopedic surgeon at Osaka University, and teamLab, a Tokyo-based IT venture. It’s not, however, an expensive and complicated high-tech app aimed specifically at health professionals. It was designed to be accessible and downloadable by anyone via Apple’s and Google’s online stores.
Sugamoto said the idea for creating the app originated from his years of clinical experience, during which time he realized that real-life 3-D movements of skeletal joints and other human parts often contradicted how they were described in medical textbooks, which are based on studies of cadavers. He was also frustrated by the fact that when a patient’s pain persisted after a surgery, such as in hip joints or the knees, he didn’t have a way to examine what was happening inside the body, aside from using X-ray images, which couldn’t provide detailed information about body movement.
“What many orthopedic surgeons have to do is the equivalent of a physician giving a diagnosis without measuring a patient’s blood pressure,” Sugamoto said.
To get a closer look at real-life body movements and a 3-D depiction of body parts, Sugamoto created what could be described as giant medical flip-books. He produced massive volumes of MRI and CT scans of around 30 volunteers, changing the angles of their necks, arms, hips and legs for each scan to capture several different images of every body part. The painstaking process took Sugamoto 10 years to complete, but when those images were finally put together, they re-created an accurate graphic of human body movements.
For IT venture teamLab, the biggest challenge in developing Sugimoto’s mass of information into an app was, according to Hironori Sugino of the company’s User Interface team, figuring out which elements of the 3-D data to lose in order to create something that would run smoothly on a tablet.
Like Sugamoto, teamLab, whose members describe themselves as an “ultra-technologists group,” recognized that the 3-D data would be useful as a reference for anyone, not just doctors. A team of engineers set to work on creating a Web-based application using Sugamoto’s information, which was followed by iPad and Android apps, all of which used the Unity game engine.
“Ten years ago, only the big companies with abundant resources could develop 3-D video games,” Koki Saito of teamLab’s Computer Vision Team said as he described how Unity software has significantly evened the playing field for video-game and 3-D interactive app creators.
Unity, which was released in 2005, and whose registered developers now number 2 million around the world, costs $1,500 for a professional license, a drop in the ocean compared with the $100,000 or more charged by other game engines. “It made it easy for anyone, even college students, to create simple video games,” Saito said.
For teamLab, the majority of whose work involves construction of E-commerce sites, the game engine enables the company to expand its other creative fields even further. TeamLab has 300 employees in Japan and China, with members from numerous fields, including computer programming, hardware engineering, mathematics, architecture, CG animation as well as art and editing — and it prides itself on blurring the boundaries between the expertise of each employee to foster innovative ideas and projects.
It has already created art installations using Unity, and according to Saito, it plans on using the game engine for more, with the most recent project being the artwork “What a Loving, and Beautiful World.” Featuring various calligraphic kanji and motifs projected on a sensor-equipped screen, the work is currently on display at the departure lobby of Narita International Airport. When a visitor’s shadow onscreen touches a kanji, the character morphs into an image of its meaning — for example, the kanji for “storm” creates a stormy view, whereas the kanji for “snow” releases snowflakes (www.team-lab.net/en/portfolio/loving/whatloving.html)
Back to the more practical level, Sugamoto said teamLab plans to update teamLabBody to make it possible to call up even more information on specific ailments particular to the areas of the body being displayed. The new version, he said, will be available within six to 12 months. Good news if you’re interested in anatomy, or if, like me, a doctor’s hand-drawn diagram is not quite enough.
TeamLabBody costs ¥2,600 in Apple’s and Google’s app stores and is available in English and Japanese. For more information, visit www.teamlabbody.com/3dnote-en/index.html.