Kinect Helps in Surgery

While some games out there might give you headaches Microsoft Research is working with a group of London super brains to enable touchless viewing and manipulation of images while performing vascular surgery with the help of Kinect.

With this tech, complex aneurysm procedures are made easier and safer because surgeons can more easily maintain a sterile environment because they don’t need to manipulate equipment and they no longer need to rely on fallible human assistance to properly manipulate the visual-aid equipment.

“Until recently I was shouting out across the operating theatre to tell someone to go up, down, left right,” explains Doctor Tom Carrell. “But with the Kinect I’m able to get the position that I want quickly – and also without me having to handle non-sterile things like a keyboard or mouse during the procedure.”

The only downside to this innovation is that you’d have to be some sort of brain surgeon to use it.

Kinect for Windows SDK 1.5 is Now Available!

This just in from the incredible whiz-kids over on the Kinect for Windows team — They’ve just released version 1.5 of the Kinect for Windows runtime and SDK.

They’ve added more capabilities to help you brilliant developers make cool things!

  • Face Tracking with a real-time 3D mesh of facial features tracking the head position, location of eyebrows, etc
  • Seated skeletal tracking!
  • Improved performance and data quality enhancements to permit RGB and depth data to be mapped together (e.g. “green screen”)
  • All new samples!
  • Many more (for full details read the Team Blog post here)

They’ve also made Kinect for Windows hardware available in Hong Kong, South Korea, Singapore, and Taiwan — with 15 more countries starting next month! To learn all 31 countries Kinect for Windows will be available in check out the Kinect for Windows blog!

Kinect in the Classroom

Re-posted from Microsoft News Center

CRAIG, Colo. – –First-grade teacher Cheryl Arnett spent much of last summer playing Kinect for Xbox 360 with her grandkids.

For fun, yes, but the 19-year veteran teacher at Sunset Elementary School in Craig, Colo., also had an agenda. A longtime lover of technology, Arnett was looking for ways to teach with Microsoft’s controller-free device. When she brought Kinect to her classroom at the start of this school year, the reaction from her enthralled students was “over the top,” she said.

First-grade teacher Cheryl Arnett introduced Kinect into her classroom at Sunset Elementary School in Craig, Colo., and has been delighted by the results, which include outstanding standardized test scores.

First-grade teacher Cheryl Arnett introduced Kinect into her classroom at Sunset Elementary School in Craig, Colo., and has been delighted by the results, which include outstanding standardized test scores.

“Bringing technology into the classroom inspires them,” she said. “Their lives are different than ours were, and just giving them a book and a worksheet is not always appropriate anymore. I’m fascinated by Kinect. There’s power for kids in things that capture their interest and are also something they can learn from. We’ve barely scratched the surface of where this can go; it’s limited only by our imaginations.”

Since Arnett’s students started using Kinect to study subjects like animals, geography, and science, she has seen improvements in their comprehension and the retention of what they learn.

Arnett is not alone in her quest – teachers all over the world are using technology like Kinect to teach their students.

Microsoft is working with schools across the U.S. that are testing using Kinect in the classroom to develop “21st century skills.” This week at the South by Southwest-edu innovative learning conference in Austin, Texas, Microsoft education experts will join a panel with other educators to explore the value of interactive gaming in learning, specifically using Kinect in the classroom.

There will be a lot to talk about, said Cameron Evans, National and Chief Technology Officer for U.S. Education at Microsoft. “Who would have thought one little sensor could turn out to be this phenomenal in the academic space?” he said.

Left to right: Computer science students Jebediah Pavleas and Jack Chang, and professor Robin Angotti have created a custom Kinect app that brings math to life by letting students graph functions like distance, acceleration and velocity with their bodies instead of pencil and paper.

Launched a year ago as a controller-free gaming device for Xbox, Kinect sales topped a world-record 8 million devices in the product’s first 60 days on the market. That made Kinect the fastest-selling consumer electronics device in history, according to Guinness World Records.

The device also has taken on a life of its own outside the living room, thanks to scientists, tinkerers, hobbyists, healthcare workers, and others who have taken the time to dream up and create unique non-gaming applications for it. In the case of education, academics are both using existing Kinect games for educational purposes and creating their own games and applications.

Microsoft, keen on supporting the creative ways people were using Kinect right after it came out, released an academic and hobbyist software development kit for non-commercial projects in June 2011. Aimng to support similar creativity in the commercial world, it released a commercial Kinect for Windows software development kit last month.

Pull all the creativity together, and you have the genesis of the  “Kinect Effect” – a term coined in the hallways and conference rooms of Microsoft to describe the device’s increasingly widespread appeal and diversity of uses. “It has a lot of people excited about what they can do today, and what they can dream up tomorrow,” Evans said.

Gaming is not a new idea in the classroom – what is new is Kinect, which offers a human user interface that creates a rich and simulated world that participants can be involved in. “We have the ability to get data behind game play to help inform instruction,” Evans said.

The potential behind bringing gaming to education is immense, he said.

The majority of time spent playing video games, the gamer is failing, he said. Yet when a player fails in a game, they come back again and again until they get it right. Bringing gaming to education could help a student who fails a math test, for example, keep trying until they get it right.

Reaching Out, Touching Math

Math, as it turns out, is one of the most abstract subjects to learn. Students learn it in books without easily seeing the concrete results of their work in the real world. That, say math teachers, makes it hard for them to retain what they learn.

Professor Robin Angotti uses her hands to move a line around a graph and watches its equation change. Angotti wants to use Kinect to change the way students learn math, making its abstract equations and concepts more approachable and concrete.

Robin Angotti (shown above), now an associate professor of math education at the University of Washington-Bothell, hopes to change all of that.

Angotti and two computer science students at UW-Bothell, Jebediah Pavleas and Jack Chang, have created a custom Kinect app to help teach students various functions of mathematics such as distance, acceleration, and velocity by letting them plot these equations on a graph in real time using Kinect and their bodies rather than just computing an equation with a pencil on graph paper.

“I was a high school math teacher for 10 years, and I knew I wasn’t reaching all of the students. I knew I was missing something,” Angotti said.

So she went back to school to get a Ph.D. in math education. Then she got “bit by the research bug,” which is how she came across Kinect and decided to create a math app that would bridge the gap between the abstract formulas and the real world her students live and breathe in.

“Math is a gatekeeper. If kids don’t get into algebra by their freshman year of high school, they’re off track to major in any kind of STEM [science, technology, engineering or math] field,” Angotti said. “It’s really interesting – data shows that math is a favorite subject when kids are younger. Somewhere in the middle school years, when they’re starting to have to abstract (when math moves from the concrete world of addition, subtraction and multiplication to the more abstract equations of geometry and algebra), we’re losing them. This piece of software makes math less abstract.”

When word got out that Angotti was researching Kinect apps for math, she was inundated with requests for more information and for advice from other teachers.


Who would have thought one little sensor could turn out to be this phenomenal in the academic space?


– Cameron Evans, National and Chief Technology Officer for U.S. Education at Microsoft

She, Pavleas, and Chang are already working on a next version of the app that incorporates a number of new features including a two player mode, so students will be able to use their bodies to explore math – and perhaps even solve the classic “two trains traveling at different speeds” story problem.

“It’s amazing. I think we’ve only scratched the surface. I see so many applications for education,” Angotti said. “We’re not just saying here’s the equation, or telling them what to do and having them repeat it. It’s a phenomenon where they can reach up and touch the equation, and move it around. All of a sudden they’re asking different questions, and there’s this sense of understanding.”

Students in the United States have notorious struggles with science, technology, engineering and mathematics (STEM) education. The U.S. Department of Labor predicts that by 2014, the U.S. will have more than two million STEM-related jobs, but there may not be enough qualified workers to fill them.

Fewer than 15 percent of current U.S. college undergraduates are pursuing degrees in science or engineering—compared to more than 30 percent in India and more than 40 percent in China.

“In a nutshell, it’s about America’s competitiveness. Our readiness for college, career, and life is just not there,” Evans said. “In some ways, it’s because the students’ engagement isn’t there. That brings with it a ripple effect that’s going not just across communities, but across generations and time.”

First Graders Without Walls

In Colorado, Arnett’s first grade class is planning a trip to Disneyland – virtual Disneyland, that is, which they will visit with the Kinect game Disneyland Adventures. But not before they learn budgeting and calculate the finances of the trip, learn some geography should they choose to drive, and even take a field trip to the airport to learn about air travel.

At the end of the project, each group of first graders will get to spend 30 minutes playing Disneyland Adventures and will have to agree together on how to spend their time in the virtual Magic Kingdom.

“It’s a tremendous amount of math, geography, and collaborative learning,” Arnett said.

At age 40, Arnett switched careers from banking to become a teacher. She’s always loved technology, but as a teacher she helps others love it too, pioneering a technology-rich classroom, school, and school district complete with smart boards, document cameras, and now Kinect.

She’s also become an advocate for innovative technology, and has traveled around the world to meet and collaborate with other teachers through programs such as the U.S. Innovative Education Forum, presented by Microsoft Partners in Learning “The world has changed. My life has changed. I have friends from all over the world I talk to on a daily basis, and it’s all because of technology,” she said. “Never would I have imagined this for myself, but these kids are going to be doing different things in their lives ahead of them, more than I can imagine right now.”

Embracing technology has not only changed her life, but her classroom. Her students are now doing a water study project with classes in Brazil, India, Saudi Arabia and Taiwan. The students use Skype to speak to children in other classrooms around the world. They watch webcasts from the Smithsonian Institute with experts, researchers and scientists who talk to them and teach them remotely on a variety of subjects.

Add it up, and technology and Kinect have earned a permanent place in her classroom.

“You couldn’t take me back,” she said. “And the teachers don’t have to know how to do everything – get the kids involved. There are a lot of things I don’t know the answers to, but I’m willing to work with them, and find the answers with them. We need to be the facilitators, the ones who make these opportunities available. This is important. Learning is real.”


GDC 2012: The Word of the Day is “Proprioception”

Today at the Game Developer’s Conference in San Francisco Double Fine’s Nathan Martz spoke about developing Sesame Street: Once Upon a Monster for Kinect.

“The ability called proprioception [From Latin proprius, meaning “one’s own” and perception, is the sense of the relative position of neighboring parts of the body and strength of effort being employed in movement] tends to be poor in children,” explained Martz, “Which means they struggle with asymmetrical gestures and frequently confuse their left and right.

“We realized we had to do was add very forgiving gestures,” said Martz. This way Once Upon a Monster doesn’t worry if the child raises her left or right hand. Determining which false negatives kids are more likely to offer and developing around it, even against common Kinect development rules, was necessary. The natural relationship with the movement of the body had to be a bit forgiving, because kids move a LOT. Sometimes kids don’t always move the way they think they want to move.

Additionally, Once Upon a Monster was designed to be as Martz explained, “Friendly to the rhythm of family life.” This means something even more dynamic than your traditional drop in/drop out multiplayer. It meant that as long as the Kinect could see someone, you keep playing.

When it came to the tried and true backbone of Sesame Street, educating kids in an entertaining way, there were additional challenges. Figuring out how to teach age appropriate lessons such as ABCs or 123’s didn’t really fit with the game. Instead, Double Fine took a “whole child education” approach that tackles larger, stickier emotional issues. Empathy, sympathy; Dreams, hopes; Wants, needs – these are the elements of the puzzles the Monsters ask kids to help solve.

“Joy is worth it,” Martz concluded. “If we want to reach a broader audience, let’s make our games about feeling more than ‘fight or flight’. We have a finite number of ideas we can express in the time that we have, and it’s incumbent on us as developers to make those ideas count.”

“Do something worth doing.”

“Kinect” With Us This Week at GDC 2012!

gdc12_logoEvery year at the time in San Francisco many of the gaming industries brightest and best gather for the Game Developers Conference. You can be sure we’ll be there talking with various space geniuses about Kinect, too!

In fact, if you’re headed there be sure to take note of two sessions of particular interest to anyone developing for Kinect”

Kinect Human Tracking – Better, Stronger, Faster: Thursday 4:00- 5:00 Room 2024, West Hall, 2nd Fl

“Kinect’s human-tracking algorithms represented significant advances in computer vision when launched just over a year ago. And luckily, the team didn’t stop there. Learn about the new features and quality improvements added since launch, as well as best practices for incorporating this new functionality into your Kinect title.”

Innovative Solutions to Gesture Detection with Kinect:  Thursday 5:30- 6:30 Room 2024, West Hall, 2nd Fl

“Many successful and innovative games use gestures as input. These games range over a wide variety of genres, platforms and input technologies, from a touch screen of a smart phone device to a full natural motion controller input device such as Kinect. Developing high quality reliant gestures is a non-trivial time consuming engineering task. In this talk we present an innovative new technology for developing high quality reliant gesture detection using machine learning. Demos, results, implementation and optimization details is discussed. Two case studies are also presented where solutions to gestures from Kinect Sports and Kinect Sports: Season Two are discussed.”

An even easier way to keep track of these awesome sessions and many more is the Microsoft at GDC Windows Phone App! This free app gives you all the information you’ll need about all the good stuff Microsoft is taking to GDC 2012 right in the palm of your hand! Check it out and download it here:

Will you be at GDC? Be sure to say hello! We’ll be there with the official KinectShare mascot, Minnie the Chihuahua. Stop and introduce yourself – we love to make new friends!Minnie01