Wekinator is a (GUI-based) application that makes it easier to connect real-time inputs to real-time outputs, with a machine learning model in the middle.

The objective of Wekinator, to paraphrase the application’s designer Rebecca Fiebrink, is to treat the training data itself as a user interface, opening the possibility of designing new types of interaction.

Wekinator makes use of “supervised learning” - you train the model by providing examples of inputs with their corresponding outputs: “if this, then that”. Using wekinator, you can quickly train a model to recognize patterns in an input stream by associating points, states, or ranges within the input to a corresponding point, state, or range in an output. Once the model has been trained, set it to run - any new input data will be mapped by the model to fit the output stream. (see this list of examples for suggestions of inputs and outputs to use with wekinator)

Wekinator receives and outputs data using the Open Sound Control (OSC) protocol - any application that can send and receive OSC can easily work with wekinator. OSC is similar to MIDI in that it is a form of control data, but it’s able to do a lot more, faster, and with finer grain. OSC makes use of UDP protocal commonly used in network communications. Messages are sent over a TCP/IP port that you choose. Messages can be sent via OSC between programs, or between machines over a network. Each application can only listen on one port at a time, but it is possible to filter messages using as many symbolic “addresses” as you care to set up: for example, http://localhost:6448/wek/inputs/01/parameter01 and http://localhost:6448/wek/inputs/01/parameter02 might represent two separate message streams sent to wekinator from another application using port 6448.

Wekinator works in one of three modes (or a combination of the three):

  • classification
    • training creates a map of input data (with any number of parameters) separated into (any number of) regions, new data is classified into these regions, sending an output message for each classification (a trigger for each activated region).
  • continuous
    • inputs are mapped to sliders, a continuous input stream (with any number of parameters) is interpolated to a continuous output stream (with any number of parameters)
  • dynamic time warping
    • given examples of a specific action as input that takes place over time (a gesture, or sequence), a specific output is triggered whenever that input action is detected. multiple types of input actions can be mapped to multiple types of outputs.