YPX

A Multi-Purpose Data Processing Tool

YPX

YPX is a powerful multi-purpose data processing tool using a visual programming paradigm for configuring filter graphs and pipelines. It is meant as a rapid-prototyping tool for interactive experiences and spaces, for testing and studying user interface technology and devices, as well as their potential combinations.

Starting off as a tiny custom config-file based command line tool for presentation of our RESi yarn (hence, “Yarn Processing”), it was further and further extended and complemented with additional functionality and a powerful UI, as well as numerous data sources and sinks, to ultimately become Yarn Processing eXtended.

YPX is meant to be a tool for fast prototyping (i.e., setting up and tuning) of small-scale to rather complex data processing methods. Using a GUI similar to well-known and widespread patching tools such as Max/MSP, Puredata, vvvv, TouchDesigner, etc., the user can easily set up directed graphs by instantiating filter nodes (“Operators”) and by configuring respective parameters and instantly see the effects at integrated live data visualizations. Existing Operators include basic algebraic operations, mathematic operations such as FFT, low- and high-pass filters, matrix rearrangements, color space conversions, logical operations, and much more.

Configurations (“scenes”) are saved to human-readable JSON files. Once configured using the GUI and ready to use, the scenes can be again loaded by an YPX instance, which also can be run headless, i.e., without UI as a console application, to save computing resources and visual clutter. To not depend on graphics interfaces (i.e., OpenGL), it can be compiled completely without UI-support using a preprocessor switch, which makes it feasible for, e.g., running a concluded processing stage on a shell-only edge-computing-device.

The idea is to have YPX in between of raw data providers, such as HIDs or other sensing devices, and target applications/frontends (e.g., a Unity3D application on a PC, an app on a Smartphone, a vehicle’s dashboard, or even another tool such as MATLAB for further processing). It would draw unprocessed data from the device, filter, and/or extract high-level information (e.g., input gestures), which will be sent to the application. To be as versatile as possible and to enable distributing components to several computing devices, it heavily utilizes networking interfaces (we prefer OSC via UDP for many use cases requiring interactive rate). Basically, any device that can send/receive data via network can be a sender/receiver. As OSC is a widespread protocol in the field of multimedia, it is easy to link YPX functionality to many existing tools (vvvv, Open Stage Control, MadMapper, etc.).

This network-centric architecture also allows for prototyping applications on mobile devices by sending data via WIFI or Bluetooth Serial, or to implement a bypass or feedback loop, e.g., by streaming data to a Python script via TCP, for processing steps requiring SciPy, then (if required) streaming the result back to YPX for capturing, etc. Data can furthermore be captured to file (CSV or binary formats) and replayed (e.g., for batch-scripted testing and controlled evaluation of different processing methods from captured input).

The tool’s source code is written in C++ and built on-top of widespread cross-platform libraries uses to ensure portability at least among Windows/macOS/Linux. Visualization is based on OpenGL, data for processing it builds upon OpenCV, so a switch to GPGPU processing is easy to do (although not implemented at this point since never required). It can easily be extended by additional processing Operators, data sources, and sinks, for example adding VRPN functionality is straightforward. The UI is built with extensibility in mind; live data visualizers are replaceable so custom implementations can be added, e.g., replacing the line graph or heatmap visualizations of an Operator type by a 3D scene rendering can be accomplished easily.

For our custom made sensor hardware, we implemented a specific protocol that we usually transmit via serial, Wifi, or Bluetooth. However, any data can basically be used quite easily, as additional data sources are straightforward to implement. So far, the tool supports the following input devices and protocols for input and/or output.

  • OSC input/output (via UDP or TCP)
  • ZeroMQ input/output
  • MQTT input/output
  • CAN Bus input/output
  • Bluetooth Serial input
  • Serial/COM interface input
  • MIDI input (via RtMidi)
  • TUIO 2.0 protocol input
  • Optitrack Natnet and Camera API input
  • Audio input (WASAPI, ASIO, DS, ALSA, CORE, etc. via RtAudio)
  • XInput Gamepad input
  • LeapMotion gesture controller input (Windows only)
  • Thalmic Myo EMG Armband input (Windows only)
  • Sensel Morph input
  • Camera input (via OpenCV)
  • Intel Realsense depth sensing camera input
  • Kinect for Xbox 360 depth sensing and full-body skeletal tracking camera input (via K4W SDK, Windows only)
  • Kinect for Xbox One depth sensing and full-body skeletal tracking camera input (via K4W SDK 2, Windows only)
  • Azure Kinect DK input (via Azure Kinect Sensor SDK, Windows only)

Furthermore, streamed data can be compressed to save bandwidth using JPEG for image data, as well as the lossless methods MiniLZO, QuickLZ, bzip2, zStd, zLib, and LZ4 for generic data.

We complement the code with sample implementations for senders/receivers: Platform IO based firmware for Arduino compatible MCUs, target application templates for Unity3D, Python, and Processing, as well as plugins for vvvv, MATLAB (mex-Plugin), and Grasshopper3D/Rhino.

YPX is currently in development (more precisely in a refactoring state for better modularity by providing a plugin-system) and will be open sourced on Github as soon as this is overhaul is finished. In case you find the project interesting and want to contribute, contact roland.aigner@fh-hagenberg.at.