TouchScript

Multitouch framework for Unity3d and Flash

TouchScript was developed by Valentin Simonov at Interactive Lab to:

  • Provide a reliable way to code multitouch interfaces on large touch surfaces.
  • Handle complex interactions between gestures.
  • Support different input methods.
  • Take into account differences between a large touch surface and an iPad.

TouchScript is much more than a handful of routines to receive touch events from different sources. Inspired by iOS, TouchScript makes handling complex gesture interactions on large touch surfaces much easier. Unity3d and Flash implementations are developed to work together in Scaleform environment.

Features

  • Easy and intuitive API.
  • Works on PC (Windows 7 and Windows 8), Mac, iOS, Android and Windows RT.
  • Doesn't require Unity Pro.
  • Supported input sources: TUIO, mouse, mobile (iOS, Android, WinRT) and native Windows touch.
  • Groups touch points into clusters on big touch surfaces.
  • Manages simultaneous gesture recognition within scene hierarchy (inspired by iOS).
  • Comes with many commonly used gestures. Easy to write custom ones.
  • Easy to test multitouch gestures without actual multitouch device using built-in second touch simulator (activated with Alt+click), TUIOPad on iOs or TUIODroid on Android.
  • It's free and open-source. Licensed under MIT license.
  • And more...


NOTE: if you got a problem using TouchScript or running examples please check the FAQ first.

Versions

TouchScript was originally developed in ActionScript 3 for Flash, rewritten from scratch in C# for Unity3d and at the moment Unity3d branch is where most of the development takes place. AS3 version as a little brother gets all the new features with a slight delay.


Unity3d

  • Written in C#, uses standard C# event system.
  • Tested with Unity 3.5.x and 4.x.
  • Handles touch points mapping from screen space to 3d space.
  • Supports native touch events on iOS, Android and WinRT.

The latest stable version can be found in Master branch on GitHub.
Develop branch is usually more up-to-date but may contain more bugs.

NOTE: Known limitations
  • Windows 7 touch recognition doesn't work in Unity IDE.
  • The library relies on C# events so it doesn't work with JavaScript.


Flash

  • Written in AS3, API is mostly the same as in Unity3d version.
  • Tested with recent Flash Player versions and SDKs.
  • Can receive touch events from Unity3d if running in Scaleform for Unity3d integration layer.
  • Works standalone, in Scaleform and in iOS/Android apps.

The latest stable version can be found in Master branch on GitHub.
Develop branch is usually more up-to-date but may contain more bugs.

Help and Documentation

If you want to know how TouchScript works please the issues section on GitHub.

If you have any questions feel free to ask them at the issues section on GitHub.

Description

TouchScript consists of a number of components to configure touch and gesture recognition plus common gesture implementations. Gesture recognition works very similar to iOS: a number of gesture recognizers (simply called Gestures) attached to scene objects (GameObject in Unity3d and InteractiveObject in AS3) receive touch point events and change state if they recognize a sequence of touch events as a gesture.

This approach makes it very easy to code complex multitouch interfaces. For example, you got a window and several buttons as its children. You want to be able to drag both windows and tap buttons. It gets complicated when user presses a button and starts dragging it. TouchScript handles this for you.


Gestures

Gestures handle touch input. They are always attached to scene objects and use their hierarchy to determine which gestures can be recognized.

There are two types of gestures: discrete and continuous. Discrete gestures when recognized set their state to Recognized and reset in a frame, while continuous ones if recognized change state to Began and keep updating with state Changed. When continuous gesture finishes it sets state to Ended.

Gestures included in the library:

  • Press (discrete) — fires when the first touch is added to an object.
  • Release (discrete) — fires when the last touch is removed from an object.
  • Tap (discrete) — fires if a user lifts off last finger from the object within certain time and movement delta.
  • Long Press (discrete) — fires if a user doesn't lift off last finger from the object for certain time.
  • Pan (continuous) — begins when user starts dragging an object and ends when the last finger is removed.
  • Scale (continuous) — begins when user starts scaling an object and ends when one or less touch points left.
  • Rotate (continuous) — begins when user starts rotating an object and ends when one or less touch points left.
  • Flick (discrete) — fires when fingers are moved fast during short period of time before lifting off.
  • Meta (continuous) — gesture which translates changes in touch points into C# events.
There's a family of Simple gestures: SimplePan, SimpleRotate and SimpleScale, which unlike other gestures with similar names are based not on clusters but only care about first two touch points. These gestures take less resources and are more suitable for small tablet-like devices.

NOTE: Meta Gesture is sort of a hack to get touch points from an object and still be in gestures hierarchy. You should never base your gesture recognition logic on Meta Gesture. Write your own gesture class if you want to handle touch points in some special way.

Examples

Unity3d

Unity3d examples can be found in \Examples\General Examples\ project, \Examples\Scaleform Examples\ project or in Examples folder in Unity package.

Examples included in the package:

  1. Basic — shows how to set up the environment and how Tap gesture works.
  2. Advanced — shows how to work with layers and different gestures in hierarchy.
  3. Hit Test — shows how to control which GameObjects receive touch points.
  4. Input — shows how to use TouchScript just to get touch point data.
  5. Basic Scaleform — shows how to use Unity3d TouchScript combined with AS3 TouchScript. Uses TestScaleformProject from Flash examples. Requires licensed or trial Scaleform for Unity3d. Just copy DLL and .cs files where they are in Scaleform examples.


Flash

Flash examples can be found in \Examples\TestProject\ and \Examples\TestScaleformProject\ projects.

Examples included in the package:

  1. TestProject — shows how to set up the environment and how most of the gestures work.
  2. TestScaleformProject — used with Basic Scaleform, handles touch events coming from Unity via ScaleformInput.

Tutorials

For better understanding how TouchScript works and how to use it please read TouchScript internals tutorial. It will guide you through most of the features you will need.

Unity3d:

Flash:

Frequently Asked Questions

  • Build fails in Unity3d Free.
    You need to delete all DLLs from TouchScript/Plugins folder except TouchScript.dll. Other DLLs contain code which is not supported in Unity3d Free.
  • Build for Windows Store/Windows Phone fails.
    You need to delete all DLLs from TouchScript/Plugins folder except TouchScript.dll. Other DLLs refer API restricted on these platforms.
  • TUIO/Windows 7 touch/iOS touch doesn't work.
    You most likely haven't added an appropriate touch source. To recieve TUIO messages add somewhere in the scene TuioInput behavior. For getting Win7 touch input add Win7TouchInput, and for mobile devices add MobileInput.
  • How to make gestures work together?
    For gestures to be recognized simultaneously you need to put one into another one's Friendly Gestures list in inspector or call gesture1.AddFriendlyGesture(gesture2); from a script. Otherwise the gesture to begin first takes all the touch points.

Version history - Unity3d:

4.1

  • Fixed a bug with Windows 7 touch in 32-bit apps.
  • Added PressGesture/ReleaseGesture.IgnoreChildren flag to listen only to touch events on the object.
  • Added TouchLayer.TouchBegan event.
  • Fixed compile warnings.

4.0

  • Added 2D support and example scene.
  • Methods which were returning RaycastHit now return TouchHit which combines RaycastHit and new RaycastHit2D.
  • Fixed an issue with levels reloading.
  • Fixed windows touch input in 64-bit builds.
  • Touch point target is now copied from RaycastHit.collider.transform, not from RaycastHit.transform since if a RigidBody is attached to object's parent RaycastHit.transform will contain parent transform not object's transform.
  • Now one camera can have several layers attached.
  • Updated OSC library used in TUIOInput.

3.1

  • Updated Scaleform integration to use version 4.2 and Unity 4.3.
  • Fixed an issue with non-standard camera viewports.
  • Fixed GUI errors in Unity 4.3.

3.0

  • Switched to Unity 4.2. Starting from this version Unity 3.5 is officially not supported.
  • Touch input handling and gesture handling functionality was separated into TouchManager and GestureManager classes. Now it's possible to use raw touch points data without any gesture recognition logic.
  • Instead of WillRecognizeWith gestures now store an array of Friendly Gestures which now has a nice visual editor.
  • TouchScript.DLL now passes WACK for Windows Store applications.
  • Added custom editors for all gestures and managers. Everything looks much better now.
  • Added Win8TouchInput which uses new Windows 8 API.
  • Updated FlickGesture and added an example for it.
  • Fixed an issue with MouseInput leaving stuck touch points (issue #10).
  • Fixed FullscreenTarget in Foreground mode when camera collider sometimes wouldn't catch touch points.
  • Fixed Transformer2D with objects without parent transform (issue #30).
  • Now PreviousPosition for still points is handled correctly.
  • Simple Scale and Simple Rotate gestures now correctly only react on the first two fingers.
  • Instead of Vector#.zero gestures return InvalidPosition if a position doesn't make sense in this case.
  • Simple Pan Gesture now has center at point if it has only one point or between first two points if it has more. To be consistent with SimpleScale and SimpleRotate getures.
  • Pan Gesture now has its center point between two clusters if there are more than one point.
  • Tap Gesture and Long Press Gesture are now discrete.
  • Clusters now return TouchPoint.InvalidPosition instead of throwing an exception if not enough points added.
  • Base Gesture now returns centroid of all touch points as ScreenPosition instead of the first point's position.
  • Removed FullscreenBackgroundTarget from the library. Use FullScreenTarget instead.
  • Removed SimpleLongPressGesture and SimpleFlickGesture.
  • Removed SimpleTapGesture, TapGesture now reports its position more accurately.
  • Renamed MinPointDistance to MinPointsDistance and made it actually matter in SimpleScale and SimpleRotate gestures.
  • Renamed TouchesCount to TouchPointsCount and Touches to TouchPoints on TouchManager.
  • Moved Win7TouchInput to separate DLL.
  • Rearranged examples.
  • Other minor changes and fixes.

2.1.0

  • Changed several API names to make more sense.
  • TapGesture now correctly reports touch position.
  • LongPressGesture now correctly handles cluster's movement.
  • Renamed Clusters to Cluster and fixed movement calculations for Flick and Tap gestures.
  • Divided gestures into simple and cluster-based ones. Simple gestures should be faster to process.
  • Added some features to Scaleform integration.
  • Touch points projection plane now has its center in collider.bounds.center.
  • Fixed the issue (#22) with namespaces and default parameters.

2.0.1

  • Fixed a crash when application using Win7TouchInput exits.

2.0

  • Upgraded HitTest mechanics. Now it skips disabled cameras and it's possible to ignore specific layers via Layer Mask. Added a special example illustrating this.
  • Win7TouchInput and MouseInput now detect if they are run on iOS or Android and don't cause any trouble.
  • More mac-friendly build process.
  • Fixed MetaGesture to work on iOS and added it to examples.
  • Made LongPressGesture and TapGesture more continuous.
  • Moved TuioInput to separate DLL so it doesn't cause troubles for Unity iOS indie users.
  • Refactored gestures and added tips to custom editors.
  • Added second touch simulator to MouseInput which is activated with ALT+click.

2.0b3

  • Major refactoring.
  • Added layers system.
  • Added custom editors to major components.
  • Setting up a scene is now much easier.
  • Added good examples and docs.
  • Integrated Scaleform.
  • Added iOS and Android support.
  • Fixed many bugs.

1.0

  • First release.

Version history - Flash:

1.1

  • Added Scaleform support.

1.0

  • First release. Cloned functionality from Unity3d repo.