This is Our Blog

June 23, 2015

Menu controlled by Kinect Avatar and oculus

Menu controlled by Kinect Avatar and oculus

This tutorial will explain how we can control Menu with Kinect Avatar.

Step 1 – Create a semi-transparent menu in oculus.

  1. Open your Scene à go to GameObject à UI à Canvas à Rename it “MenuCanvas” à set component values as below image

  1. Select MenuCanvas à right click à UI à Panel

    Add Component à Layout à Vertical Layout Group

  2. Now Add buttons to make Menu items

    Select Panel à right click à UI à Button.

    Go to Image (Script) à Image Source (Change UISprite to Background)

    Note – Customize your Buttons UI through Button (script).

  3. Duplicate your Button to five times. Now you have total six buttons as Menu items.

    Set buttons text as below image

  1. Write script for button event.

    Create a C#‎ Script named “ButtonsEvent” and add below code to your script.

        public void btnClick1()
            Print("Button1 Clicked");
        public void btnClick2()
            Print("Button2 Clicked");
        public void btnClick3()
            Print("Button4 Clicked");
        public void btnClick4()
            Print("Button4 Clicked");
        public void btnClick5()
            Print("Button5 Clicked");
        public void btnClick6()
            Print("Button6 Clicked");

    Now add ButtonsEvent.cs to Panel.

    Attached function to every button OnClick().

    Ex- attach function to Button1.

    Attached function to rest buttons.

    Step 2- Setup your Kinect and oculus to project.

    If your project do not have Kinect and oculus assets, please read our blog

    Step 3- Add VRGUI asset to project

    VRGUI asset is used to make GUI component visible in oculus. You can download asset and read more from

    Note- First login with your oculus account then you can download the attached file.

    Step 4- Menu always in front of the users head Oculus.

    Expand OVRCameraRig à Drag MenuCanvas inside CenterEyeAnchor

    Then select MenuCanvas and Deactivate.

    Step 4- Kinect Avatar gesture to perform actions like hide, show menu and click.

    Gesture for Hide and Show Menu

    Select Kinect Camera à SimpleGestureListener (open Script component in your editor)

    Replace your GestureCompleted function with below code

        public GameObject menuPanel;
        public bool GestureCompleted(long userId, int userIndex, KinectGestures.Gestures gesture, KinectInterop.JointType joint, Vector3 screenPos)
            string sGestureText = gesture + " detected";
            if (GestureInfo != null)
                GestureInfo.guiText.text = sGestureText;
            progressDisplayed = false;
            if (gesture.ToString() == "SwipeDown")
       = true;
            else if (gesture.ToString() == "SwipeLeft")
       = false;
            return true;

    Go to unity scene à Select kinect camera à go to SimpleGestureListener script component à assign MenuCanvas to Menu Panel.

    Mouse Controlled by Avatar Hand

    Select kinect camera à go to InteractionManager Script component à Set enable to Control Mouse Cursor.

    By default cursor is controlled by both hands and fire one click event in every 2 seconds, you can customize these setting like click event fire only when hand grip.

    Now play Application.

    When you’re Avartar get controlling by kinect then you can perform below gestures(tasks)

    1. Swipe Down you hand to visible menu.

    2. Swipe Left you hand to dismiss menu.

    3. Move hands to focus menu items.



Would you like to share your thoughts?

Would you like to share your thoughts?

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.