VR and Leap Motion: Early lessons learned

Published April 11, 2015
Advertisement

Leap Motion:
This is actually really finicky to get working properly with UE4. I don't know what exactly the issue is, but I'm experiencing some pretty big latency between my hand gestures and the device response.

Issues so far:
USB Speeds: I was using the USB port on my computers front panel display. I guess it is slower since its being reported as going through a USB hub.
Sunlight: The SUN! It actually interferes with the hardware. It's got an infrared sensor which it uses to track motion. I was testing my leap motion on a sunny day here in Seattle, and I'm sitting right next to the window. The shade was down, but not blocking 100% of the light. I was experiencing lots of lag, but then things suddenly got a lot more responsive. I looked outside and the sun happened to be temporarily behind a cloud. Then I made the connection.
Latency: Even if you aren't running an application which is using the leap motion device, it is on and capturing data. I didn't realize this for a while and wondered why my computer was acting sluggish. The computer felt like it was constantly getting page file misses and had to perform a page fault every time I switched to a different application. It would take a second to get going again every time I switched to a different application. The solution is to turn off the leap motion controller from its task bar when you're not using it. It would be really nice if I could just turn it on and off via a function call in my game...

Oculus Rift DK2:
Resolution: So, the screen pixel resolution isn't very good. If you have smallish objects off in the distance, it'll be a blur of pixels. I think this'll be vastly improved on the consumer release, but it does remind me a bit of playing old school games in 800x600. The image just isn't crisp like you'd expect on a monitor.
Motion Sickness: I'd heard about this and thought that the people experiencing it were a bunch of weaklings, but I wouldn't get it. I started getting it. I think I know what causes it. So, the VR headset gives you a sense of "presence" in the game world. Your brain thinks your body is in it. But then you start moving around. Your physical body doesn't feel the forces of the movement, so its barely confused. That barely adds to the motion sickness, but it probably adds up over time. I found what really causes the motion sickness is jumping, particularly the landing part. I think what's happening is that my brain "gets" its spatial representation in 3D space, and then it projects its own experiences with how movement works. When I jump, my brain is like, "Okay, we're going up, and then we're going to come down. No problem... I got this." But then, when your character finally hits the ground, the slowdown is immediate. The brain was doing some physics momentum calculations to predict where its going to be spatially and already placing itself spatially in that imaginary predicted position. But the game stops you suddenly, and the visually reported position is off from the spatially predicted position, so there's a mismatch. The brain has to quickly recompensate for what its actually seeing vs what it thought it should see, and it sort of has to do a really rapid linear interpolation between its expected position to its actual position. This mental lerping process is where the motion sickness comes in. Anyways, I think the best practice is to limit jumping, but if you have to do it, then do it with more realism, and let the player camera do a rapid slow down rather than coming to an abrupt stop. You can sell it better to the player by having a character go through a bit of a crouch animation when they hit the ground, and then springing up. Test it out though, I may be wrong.
Hot heads: The DK2 is a bit uncomfortable. I find that the nose section is sharp against the bridge of my nose. But, its a minor issue. There's another issue that kind of comes up: Heat dissipation. If you have long hair, then put on the VR headset, then add on headphones, your head will get warm in about 10 minutes. If you're in a warm climate, this may be a bigger issue.
Scale: The VR headset conveys scale a lot better than a flat screen. I made what looks like a little hill in my game editor. On a flat screen, it looks kinda small. But if you wear a VR headset, the hill actually looks pretty big and has some steep parts. Its more than just a mound of dirt, it's something to climb. You can really use this fact to great effect with creating environments. Those trees start to look a lot taller and bigger now smile.png
Sound: You gotta convince someone that the thing they're seeing in VR is real. Believable sound effects can really help with this.

Overall, I think that if you're building a VR game, you should aim for 10-15 minute VR sessions. In my game, you'll be doing the tactical battles in VR. These battles should last 10-15 minutes, so that players can take a break if they need one. Cool off their heads, reorient/recalibrate their balance, take a breather, etc.


Leap + VR:
The VR problem is that you can't see your hands on the keyboard. With the LM sensor, you can toggle between viewing the virtual world and viewing the real world through its built in IR camera. This is a quick way to see your hand positions on the mouse and keyboard and any IR interference.

3 likes 2 comments

Comments

Navyman

Not having any experience with the VR or Leap I am happy to read about so of the pains to use the items.

April 12, 2015 08:17 AM
Orymus3

Judging by the posting rate, I'd be inclined to believe that adding VR to your project has dragged a wind of motivation along?

"Overall, I think that if you're building a VR game, you should aim for 10-15 minute VR sessions. In my game, you'll be doing the tactical battles in VR. These battles should last 10-15 minutes, so that players can take a break if they need one. Cool off their heads, reorient/recalibrate their balance, take a breather, etc."

I love that you're taking this into consideration. I think a lot of the buzz around VR currently ignores the user experience. As you state, the first 10 minutes of a VR experience are awesome. Though the awesomeness fades slowly, the actual discomfort does increase quite fast, and it's good that your design will take this into account to maximize the experience.

Keep it up!

April 17, 2015 05:03 PM
You must log in to join the conversation.
Don't have a GameDev.net account? Sign up!
Profile
Author
Advertisement
Advertisement