The Wekinator: Software for using machine learning to build real-time interactive systems
by
Rebecca
Fiebrink, Dan Trueman,
and Perry
Cook
In short, you haven’t heard about Wekinator because I was nose-deep in dissertation work until quite recently. The Wekinator is software I built during my recent PhD at Princeton, and it’s meant to enable composers and musicians to more easily use machine learning to create interactive systems. (Please see the website for more info: http://wekinator.cs.princeton.edu/)
The software is currently in a stable state, and it’s usable by anyone who wants to control ChucK, Processing, Unity, Ableton, or pretty much anything else that can be controlled by a stream of OSC messages. These audio/video/etc. systems can be controlled by gesture (Kinect, Arduino, USB game controllers, webcam, …), audio (e.g. using ChucK audio feature extractors, Max/MSP analyzer~, etc.), or anything else that can extract information about human actions in real-time and pass it to Wekinator via OSC.
The software will continue to evolve. There are some architectural changes which must happen soon, but if anyone is looking to help with development, there are a few areas where I see room for collaboration:
– UI (to keep this cross-platform, my plan is to stay in Java, using JavaFX + processing; that said, the UI needs an overhaul)
– Documentation (READMEs, FAQ, and in-app help)
– Examples (e.g., putting together a nice repository of code examples of ChucK, Max/MSP, Ableton, Processing, etc. being driven by Wekinator)
– Determining what is necessary for tighter integration into Max/MSP (e.g. using Jamoma framework), with working examples for our Max-loving friends
Additionally, I’m very interested in hearing suggestions, bug reports, and feature requests from people who are actively using Wekinator. From the beginning, the software has been driven by the ideas of people actually using it to make music, and it’s become a much better project as a result.
Lastly, if this project interests you at all, please consider joining the Wekinator mailing list at http://groups.google.com/group/wekinator-users?pli=1. That’s the best forum to share your ideas, bugs, questions, etc.
Best,
Rebecca