Naima’s First Flight; why this interface exists and why it’s important (to me)
(first, an apology to the MIT Media Lab. for not being able to demonstrate my latest Exo-voice construct on account of not having a working knowledge of the wifi modules I chose to use. with that said, I plan to come back that way with the evolved systems to share why it may not have been such bad thing for the concept to have failed at that exact moment.)
soooo….hey ! its been a while. in October of last year, I ended my workshop residency at 5200 Chene st (thank you again Ben and Vanessa, for allowing me to inhabit your space all summer) and came back to where I have been living here in Highland Park and began pondering something I had discovered slightly earlier in mid September; that
all the fingers have a direct relation ship with the thumb.
I angled the pinky key so that it was facing the thumb position, like grabbing a baseball, rather than facing down, like typing on a keyboard. this single tweak changed the whole feel of the system. the fingers seemed to simply “fall” where they were supposed to. it was a completely new experience!
jumping back to october, newly relocated back at what we will call the “home lab”, I was trying to make sense of this revision. it uncovered the optimal hand positioning for the hand units, but then I realized that the thumbs position out to the side of the hand was now obviously wrong. a few days were spent investigating changing the thumb positon AGAIN when it became obvious that this hand unit design was not gonna make the cut.
the previous hand units were designed for the previous-perfect hand positoning and for its modularity. one thing I have learned throughout this process, is that you can not get too attached to a particular form because what is perfect today is tomorrow problem to solve. as such I went to the note book and began playing with this thumb to finger relationship and came up with this design sketch.
around that same time, 3d printed circuits were haunting my dreams. I had spent most of the summer rebuilding my reprap to extrude dual materials, for the purpose of creating integrated circuits inside all of my future designs. so with the new design, there was already an inbuilt 3d printed circuit and sensor conceptualization, which are represented in the sketch as those small black lines. this was the most efficient touch/pressure sensor I could imagine at the time.
And so it begins
So, at the crib, ive got loads of materials; f-electric conductve, taulman nylon and ninjaflex flexible filaments, a half dozen ESP8266 radios (more on these later) and some atmega 328p IC’s. the goal of the winter period was to evolve this new construct as the basis of the release version of the exo-voice system. that system would be even more 3d printed, more ergonomic and use less expensive components that were also more powerful than the currently used ones. designed properly the exo-voice system would convey its gestural trajectories by virtue of the design, be easier to assemble, and cost MUCH less; going from a total of about $350 to about $60; TOTAL!
You can see here how hand position can convey gestural axial position by using the flat side as a sort of meter. in this pic the left hand is all the way to the extreme of the xaxis and the right hand is in the very middle. it is easy to see from the front or the back, as the pilot.
most of october and november was spent experimenting with printing circuits and sensors and testing them with LEDs
By december, I felt I was ready to begin work on the new hand units but felt “distracted” by the existing (working, refined) exo-voice. it was going to be harder to go where I needed to go with the other system there acting as a net when (not if) I failed, so I made a choice; send it to one of my contributors (who shall remain nameless). this solves two problems simultaneously; 1. its not here influencing my design decisions or giving me an excuse to just use it instead, and 2.being a sort of “soft” start to campaign fullfillment, I can get feedback from one of my contributors on how approachable the concept is currently and use those notes to refine the current system. in addition, I get it back once I have the release system ready to swap out with him later.
As you can see in these photos (there is much more detailed information in the video), design concessions had to be made to get it all flowing. I had been all gung-ho about printed circuits but I found their conductivity to be too low. for sensors, printed conductive traces are great and revolutionary. sensors are now a design element. the entire design is now much more maleable thanks to this. but they are shitty for circuits. the electricity gets from point A to point B but at a greatly reduced current density. I decided to save that skirmish for a future date and went with wire.
the benefit of forgoing printed circuits first was that I could just run branches of anode and cathode wire to wherever they needed to be.
the Leds are brighter than previous leds I have used so only 2 of them were needed ; one for system feedback and one for edit parameter feedback. their placement is much more fortunate and also help convey axial postioning in addition to the “L”-shaped design.
note at this point; this stage was reached 2 days before I was to fly to Boston to present and demonstrate the latest aspects of the system at MIT. everything in these pics worked wonderfully except one thing…
The ESP8266 wifi tranciever
I do not believe there is anything wrong with the ESP. just as I have learned dozens of times before, the lapse is/was in my over, under and inner-standing of the module. there are so many ways to approach programming it now, including NodeMCU, Arduino IDE, Python, as well as AT commands. there were 2 days at this point to figure it out and made great progress!! but not enough progress to get it working in time for my demonstration
although I always warn clients that I have no idea what I will be doing on the night of an event, it will usually at least work slightly or catch fire (has happened) or something. but on this night, the system still existed in a future that is still pulling me toward it. it’s the equivalent of not being able to speak so it was very frustrating but somehow, necessary and very non-coincidental. my router died earlier that day as well and none of the very helpful staff or students at MIT could get anything out of it either, so something was in the air, I feel. and that something usually pushes toward more novel abstractions so rather than trying to brute force it to work I meditated on it for the week spent in new york and in philly , before flying to morocco 10 days later.
while in New york visiting friends, I was haunted by this inability to perform. when not meeting up with friends, I was studying basic programming (arduino IDE uses a variant of C++ as its language)and kept getting close but finally realized that it was going to be a big gamble as to whether I could get the wifi going properly, then refine the construct before the casablanca event on feb 4th.
so I shifted gears; what if I took some of the spare parts I brought with me, and designed a “something” playable? the limiting factors were
- arduino fio
- an RN-XV radio tranciever
- some LEDs
- an accelerometer, taken from one of the new hand units
- some wire
- an old air pressure sensor
- NO touch sensors-after 4 months of experimentation, I knew how to make working printed pressure sensors and switches.
a single arduino has 8 analog inputs and 12 digital pins that can be input, output or pwm (pulse width modulation) which mean they can output a dimmer-like signal. I use these to control LEDs.
the first idea was this-a playable mask
it would be played by placing my fingers on the sides of my face and blowing into the mouthpiece. I may still try this one day but under the crunch I was feeling, I quickly scrapped this idea but not before I got this from it
when laying out the fingering, I realized that rather than using all the analog inputs for finger keys, meaning no analog inputs for breath and accelerometer, I could use 1 analog in for breath, 2 for accelerometer and 4 for the fingers, namely the thumb and index finger on each hand, then use 3 digital switch inputs per hand for the keys. those 4 analog inputs would respond to finger pressure, allowing them to be use as a key when pressed lightly and as a edit mode function when pressed hard. (imagine holding a tiny sandwich with just the index finger and thumb of both hands…this is the mental construct.) those 4 would form a 4 bit array by typing 4 bit values in, there would be 15 different finger combinations that could be used to control things. yaaaaay!!! it ends up being an accidental gestural wind controller!!
the next stage after this was to scrap the mask idea a go for a horn like design.
same fingering concept, using the same components. the accelerometer would be in the end and I could turn the horn left to right or up and down to use it. in addition, the octaves would be controlled by the accelerometer, adding to the new simplicity. but as I printed the parts for this design, I realized that it was going to use A LOT of material and a lot of wire and was going to be bigger than my previous wind controller, which was instantly not interesting. I didn’t escape from wind controller-dom just so I could build a shittier one later.
so by now I’m in Philly staying with a friend who goes by the name grouchofractal and meditating on this issue. efficient use of material. efficient ergonomics. 4 bit array. hmmmm….then something blinked into view just before heading to the airport .
from the player position, the 4bit array would feel like this.(remember the tiny sandwich reference above?)
the circuit became much simpler and cleaner, with the accelerometer down at the tip, the pressure sensor embedded in the upper part of the mouthpiece with its own embedded LED and of course the new sensors.
it is really great to realize that all the design iteration practice of the last few years can be called to service so easily and quickly.
I arrived in Casablanca and while going thru the exit scanners before exiting the airport, gave a little presentation to the customs officers who were more curious than worried about the bag full of gadgetry I was coming into their country with.
the driver and I had a wonderful conversation consisting of mouth noises, single words and hand gestures on my way into the city, which was very interesting and WARM. I was driven to meet up with Michy Mano, the man who arranged for me to come and play for the opening of a place he was opening, Mano Club. we hung out for a while and had a really interesting conversation about many things and he played me some of his music and music he likes and during this exchange I felt a strong resolve wash over me and announced to him that I was going to build the system I planned to play at his venue opening, while I was there in Casablanca. this was the sketch I made while we were chatting
he kinda looked at me to see if I was kidding and when I didn’t laugh, he expressed that “well, at least you have your other instrument, just in case…” to which I replied, “um, not really…I don’t have anything that works right now…” it is a moment like this that having a camera around would have been perfect. he went a bit blank for a few seconds- I don’t think he could figure out if I was crazy or joking because we were just meeting for the first time-but I think he chose to believe I wasn’t crazy for the moment.
as soon as I arrived at the beautiful penthouse they set up for me to stay in, I set my lab up and got straight to work
the idea translated extremely well!! the only really design fart was not putting support underneath the keys. other than that, it worked exactly how I envisioned and went from sketch to working system in about a week.
it is not without quirks. I taped conductive fabric to my fingers to make the keys and they are extremely sensitive so I was not able to use them properly at the event. instead, I made it a completely gestural instrument with minimal looping abilities. the pd patches for my exo-voice were easily adapted to this system. it is played by simply blowing and tilting the head either side to side or up and down.
the most profound aspect of this new system are that it is a full gestural breath controlled wireless interface on a single arduino!
it uses printed sensors and printed switches and can literally be played by anyone that can move their head! .not to mention that it is much more easily printed than any of my previous designs. and because the sensors are printed, in this current state it would cost about $80 to make. when I have the esp8266 working and use ebay bought arduino pro mini’s, this system can be built for about $17!!
this is the seed constructive been trying to design for years but was asking the wrong questions (a seed construct is simplest expression of the idea) and because there is one accelerometer tied to head movement and breath there is the completely surreal feeling of being the snake charmer and the snake simultaneously. it feels like blowing into a tiny stargate. one blows, and just after blowing is an LED, which feels like you are blowing into the LED directly, then there is the 4bit stage of interfacing and then next is the note interval sequencing (either 8bit fingerings or sax fingerings) with an associated other RGB LED it is a surprisingly hypnotic instrument to play and will serve as the basis of the workshop build for the upcoming sonic fractal matrix workshops in the spring!
Her name is Naima.
I was referencing a very famous coltrane song when I thought it up, but then was told that Naima is a moroccan name, which reconfirmed that the events of the previous weeks had meant to culminate in the birth of this form because I had not been thinking along these lines at all, and it took a crashed performance to serve as the catalyst.