Loading Events

An Introduction to Visual Sensors

Introduction to Kinect data processing in TouchDesigner for interactive visuals and installations.

Course Instructor: Masha Rozhnova

Course level

Attention: This workshop is Windows only. 

TouchDesigner is a tool that gains popularity among musicians who want to create visuals for their music and do audio-visual experiments. In this workshop you will be guided through the integration of visual sensors into the TouchDesigner environment. We will work with Kinect in particular but the techniques will be applicable to other sensors as well.

Those without a Kinect sensor can follow the course by using provided recordings of sensor data. You will be able to play them back via Kinect Studio software, which will act as a sensor input for TouchDesigner. The main requirement for this course, however, is a Windows machine, as Kinect Studio is only supported on this platform.

Workshop will be led by Masha Rozhnova & Dániel Molnár

Function Store (Dániel Molnár) is a Berlin based artist and developer creating interactive installations and audio-visuals.

He has a software engineering background, focusing on TouchDesigner and related tools for the past two years and recently committing full-time to it.

He has interactive installations in various locations around Dubai in collaboration with Hybrid Xperience, is working with musicians to provide live visuals and digital content, and has a YouTube channel where he shares some of his secrets.

What you'll learn

  • Operate Kinect Studio software
  • Explore data types provided by visual sensors
  • Understand how to use Player index, Skeleton and Point Cloud data
  • Create interactive visuals driven by sensors

Requirements

  • A Windows 8 or 10 PC with Internet connection
  • Kinect SDK 2.0 installed
  • A three button mouse
  • TouchDesigner version 2021.15800 installed (non-commercial suffices)
  • A webcam and mic
  • A Zoom account

Course content

  • ---- Lesson 1: Setup
  • ---- Lesson 2: Tracking Joint Positions with Kinect CHOP
  • ---- Lesson 3: Tracking Absolute Bone Rotations with Kinect CHOP
  • ---- Lesson 4: Using Depth Data from Kinect TOP
  • ---- Lesson 5: Using Point Cloud Data from Kinect TOP
  • ---- Q&A 1: Using Blob Track TOP with Depth Data
  • ---- Q&A 2: Filtering out unknown Value Noise
  • ---- Q&A 3: Where to buy Kinect V2

Who is this course for

  • For students who want to learn visual sensor systems
  • Getting to grips by using visual sensor data creatively
  • Students who will be utilizing data streams
  • Learn data input

Useful links

Downloads - Kinect SDK 2.0 TouchDesigner

About the workshop leader

Dancing Pixels (Masha Rozhnova) is a London based artist who creates audio-visual performances and videos for musicians. For the past three years she has been using TouchDesigner as the main tool for content creation, show control and as an engine to enable interaction with the audience. She performed at Live Performers Meeting in Rome, at New River studios and Crux events in London.