Tag Archives: programming

AppHack-e :: ‘Appcopter’ toy helicopter, hacked. Part II: Flying robot

In the previous post of this two-part series, I revealed the process by which the binary control signal for the ‘Appcopter’ toy could be deciphered. This was the first step towards the ultimate goal of commanding the helicopter to perform a sequence of predefined manoeuvres, like the well known educational toy robot ‘Turtle‘.

The remaining task is to write a program capable of generating an audio signal that looks like the image below. Chattering the signal out of a computer’s line out port and into the transmitter will send the toy off doing whatever I tell it to, so long as the parameters are within bounds – “fly, my pretty, fly!”.

Throttle at 10%

Generating the signal

I immediately set out to use Processing (a Java based programming tool) as my programming tool of choice; it’s a free, cross-platform tool that’s pretty easy to get into. Audio support ‘straight out of the box’ isn’t quite adequate for the purposes of easy audio manipulation, so I opted to plug in a third party library called Minim in order to manage my requirement for dynamic audio generation.

Brace yourselves for some code – there’s no way to make this section any easier than simply posting the full code with explanatory comments. Feel free to use this source as you wish (evil robot swarms excluded).


// IMPORTS

// requires Minim audio library: http://code.compartmental.net/tools/minim/
import ddf.minim.*;
import ddf.minim.signals.*;
import ddf.minim.effects.*;
// Java's AudioFormat is required by Minim's createSample() Function
import javax.sound.sampled.AudioFormat ;

// VARS
Minim minim ; // minim instance
int outputSampleRate = 44100 ; //audio sample rate (hz)

// arrays storing prefabricated signal elements - a full signal is built from
// assemblies of these elements in the correct order for a particular message
float[] sig_open ;
float[] sig_0 ;
float[] sig_1 ;

// affects how quickly one program value becomes another - rather than change e.g. throttle abruptly, smoothly ease it from one value to the next
float ease = 0.75 ;

// 'current' control values - these are the values that are actually sent to the transmitter. They are based on the desired value contained in the current instruction, modified by easing
int cThrottle ;
int cPitch ;
int cYaw ;
int cTrim ;
//

// signal is being generated and transmitted during main update
boolean signalOn = false ;

// time at which the program began (used to compare time to next instruction)
int programStartTime ;

// a modifible audio sample to host the signal data
AudioSample signal ;

// In the same way that a washing machine has a program - 'prewash, main wash, spin' - I want to be able to say 'take off, fly forwards a bit, turn left', etc. I use a list of Instruction objects, with a variable pointer to the current instruction. the suffix '_ch1' indicates that this could be made into an array of instruction arrays, thus supporting multiple airbourne agents simultaneously controlled by one computer
int programIndex_ch1 = 0 ; // current program index (1)
Instruction[] program_ch1 = new Instruction[0] ; // program instruction set (1)

// CONSTANTS
int MIN_THROTTLE = 0 ;
int MAX_THROTTLE = 127 ;
int DEFAULT_THROTTLE = 0 ;
int MAX_PITCH = 0 ;
int MIN_PITCH = 63 ;
int DEFAULT_PITCH = 32 ;
int MIN_YAW = 0 ;
int MAX_YAW = 31 ;
int DEFAULT_YAW = 16 ;
int MIN_TRIM = 0 ;
int MAX_TRIM = 63;
int DEFAULT_TRIM = 31 ;
int DEFAULT_CHANNEL = 1 ;
int DEFAULT_SPROG = 2 ; // in the last post I had no idea what this was - it apparently controls the on/off state of the the bright white nose light, but may also be something to do with resetting the helicopter control board between flights!
//
int throttleBits = 7 ;
int pitchBits = 6 ;
int yawBits = 5 ;
int trimBits = 6 ;
int channelBits = 2 ;
int sprogBits = 2 ;
//

// INITIALISATION
void setup()
{
 frameRate(6) ; // feels about right: repeats the full signal every 6th of a second
 size(256, 256); // any old window size - doesn't really matter since there's no UI

 minim = new Minim(this); // make a Minim audio instance

 int i = 0 ; // a counter

 // 0, 1 prefix: each signal value (0 or 1) is preceeded by a fixed length output close to zero. In this block of code I create this data 'building block', in such a way that it can be included in order with other building blocks, and a full signal thus made
 float[] sig_prefix ;
 float prefixSignalDuration_ms = 0.41 ; // the duration of the signal element
 int prefixSignalNumSamples = round( outputSampleRate * (prefixSignalDuration_ms/1000.0) ) ;
 sig_prefix = new float[prefixSignalNumSamples] ;
 for( i = 0 ; i < prefixSignalNumSamples ; i++) sig_prefix[i] = 0.01 ; // set a value very close to zero

 // open: construct the 'on' signal building block, which preceeds the data values in every full signal
 float openSignalDuration_ms = 1.6 ;
 int openSignalNumSamples = round( outputSampleRate * (openSignalDuration_ms/1000.0) ) ;
 sig_open = new float[openSignalNumSamples] ;
 for( i = 0 ; i < openSignalNumSamples ; i++) sig_open[i] = 1.0 ; // the 1 signal data (set to -1 for MacOS?)

 // the '0' value building block
 float zeroSignalDuration_ms = 0.4 ;
 int zeroSignalNumSamples = round( outputSampleRate * (zeroSignalDuration_ms/1000.0) ) ;
 float[] tSig_0 = new float[zeroSignalNumSamples] ;
 for( i = 0 ; i < zeroSignalNumSamples ; i++) tSig_0[i] = 1.0 ; // the 0 signal data (set to -1 for MacOS?)
 sig_0 = concat( sig_prefix, tSig_0 ) ; // prepend the 0, 1 prefix building block from above

 // the '1' value building block
 float oneSignalDuration_ms = 0.8 ;
 int oneSignalNumSamples = round( outputSampleRate * (oneSignalDuration_ms/1000.0) ) ;
 float[] tSig_1 = new float[oneSignalNumSamples] ;
 for( i = 0 ; i < oneSignalNumSamples ; i++) tSig_1[i] = 1.0 ; // the 1 signal data (set to -1 for MacOS?)
sig_1 = concat( sig_prefix, tSig_1 ) ; // prepend the 0, 1 prefix building block from above

// make a container for the full signal data - an assembly of the building blocks above in whatever order we choose. Length is some arbitrary value big enough to contain the entire signal
int nullSignalNumSamples = 2000 ;
float[] nullSig = new float[nullSignalNumSamples] ;
 for( i = 0 ; i < oneSignalNumSamples ; i++) nullSig[i] = 0 ; // set the full signal to zero

// build our audio sample, using our 'nullSignal' data source to drive it
 AudioFormat signalAudioFormat = new AudioFormat( outputSampleRate, 16, 1, true, false ) ;
 signal = minim.createSample( nullSig, signalAudioFormat ) ;

 // That's the building blocks ready: now use them to set up our custom helicopter control program!
 initProgram() ;
}

void initProgram()
{
 // as the battery drains, the appcopter loses power - it needs proportionally more throttle to climb to the same altitude, etc. tScale value allows uniform tweaking of the full program to keep the helicopter behaving consistently throughout multiple runs, across the entire range of the battery's charge
 float tScale = 0.5 ;

 float t = 0 ; // time offset

  int myTrim = DEFAULT_TRIM - int(11 * tScale) ;
// my heli needs this - one too many crashes I think!

  // spin up and stabilise - send '0' to the 'sprog' param briefly; seems to be the way that the proper Appcopter app does it (reset?)
  program_ch1 = appendInstruction( program_ch1, t, 0, DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, 0 ) ;
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.3, DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  //

  // fly, my pretty, fly... Draw a square.

  // lift off and stabilise
  program_ch1 = appendInstruction( program_ch1, t+=1, 1 * tScale , DEFAULT_PITCH, DEFAULT_YAW, DEFAULT_TRIM, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.6, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  //

  // forwards - pitch forwards for a time
  program_ch1 = appendInstruction( program_ch1, t+=1, 1 * tScale , MAX_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  // strip momentum and stabilise - pitch briefly back to arrest forwards momentum and return to hover
  program_ch1 = appendInstruction( program_ch1, t+=1.5, 1 * tScale , MIN_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.3, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // turn 90 - yaw full for a brief time
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.9 * tScale , DEFAULT_PITCH, MIN_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.5, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // forwards
  program_ch1 = appendInstruction( program_ch1, t+=1, 1 * tScale , MAX_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  // strip momentum and stabilise
  program_ch1 = appendInstruction( program_ch1, t+=1.5, 1 * tScale , MIN_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.3, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // turn 90
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.9 * tScale , DEFAULT_PITCH, MIN_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.5, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // forwards
  program_ch1 = appendInstruction( program_ch1, t+=1, 1 * tScale , MAX_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  // strip momentum and stabilise
  program_ch1 = appendInstruction( program_ch1, t+=1.5, 1 * tScale , MIN_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.3, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // turn 90
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.9 * tScale , DEFAULT_PITCH, MIN_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.5, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // forwards
  program_ch1 = appendInstruction( program_ch1, t+=1, 1 * tScale , MAX_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  // strip momentum and stabilise
  program_ch1 = appendInstruction( program_ch1, t+=1.5, 1 * tScale , MIN_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.3, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // turn 90
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.9 * tScale , DEFAULT_PITCH, MIN_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;
  program_ch1 = appendInstruction( program_ch1, t+=0.5, 0.9 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // landing - throttle down to allow safe descent
  program_ch1 = appendInstruction( program_ch1, t+=1, 0.55 * tScale , DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, DEFAULT_SPROG ) ;

  // shutdown - set 'sprog' to 1, again seems to be the way appcopter app does it, but seems maybe unnecessary?
  program_ch1 = appendInstruction( program_ch1, t+=1.5, 0, DEFAULT_PITCH, DEFAULT_YAW, myTrim, DEFAULT_CHANNEL, 1 ) ;
}

// construct an instruction and put it into the control program array
Instruction[] appendInstruction( Instruction[] program, float time_s, float throttle, int pitch, int yaw, int trimOffset, int channel, int sprog )
{
  Instruction instruction = new Instruction( time_s, throttle * MAX_THROTTLE, pitch, yaw, trimOffset, channel, sprog ) ;
  program = (Instruction[]) append( program, instruction ) ;
  return program ;
}

// the main update loop
void draw()
{
 if( ! signalOn ) return ; // don't do anything if the signal is off

 // otherwise...
 readProgram() ; // fetch the latest instruction
 sendData() ; // transmit it as audio
}

void readProgram()
{
 float currentTime_s = float(millis() - programStartTime) / 1000.0 ;

 int nextInstructionIndex = programIndex_ch1 + 1 ;

 if( nextInstructionIndex < program_ch1.length )
 {
 Instruction nextInstruction = program_ch1[nextInstructionIndex] ;
 if( nextInstruction.time_s < currentTime_s )
 {
 programIndex_ch1 = nextInstructionIndex ; // next instruction is now due: step the program index along
 Instruction instr = program_ch1[ programIndex_ch1 ] ;
 println( "Set instruction: @time: " + instr.time_s + " throttle: " + instr.throttle + " pitch: " + instr.pitch + " yaw: " + instr.yaw + " trimOffset: " + instr.trimOffset + " sprog " + instr.sprog ) ;
 }
 }
}

void sendData()
{
 float[] buffer = signal.getChannel(1) ;

 int offset = 0 ;

 Instruction instr = program_ch1[ programIndex_ch1 ] ;

 // ease
 cThrottle += (instr.throttle - cThrottle) * ease ;
 cPitch += (instr.pitch - cPitch) * ease ;
 cYaw += (instr.yaw - cYaw) * ease ;
 cTrim += (instr.trimOffset - cTrim) * ease ;
 println( "Sent data: Throttle: " + cThrottle + " Pitch: " + cPitch + " Yaw: " + cYaw + " Trim: " + cTrim ) ;
 //

 // open
 offset = writeData( sig_open, buffer, offset ) ;

 // throttle
 offset = writeValue( throttleBits, cThrottle, buffer, offset ) ;

 // pitch
 offset = writeValue( pitchBits, cPitch, buffer, offset ) ;

 // yaw
 offset = writeValue( yawBits, cYaw, buffer, offset ) ;

 // trim
 offset = writeValue( trimBits, cTrim, buffer, offset ) ;

 // channel
 offset = writeValue( channelBits, instr.channel, buffer, offset ) ;

 // sprog
 offset = writeValue( sprogBits, instr.sprog, buffer, offset ) ;
 //offset = writeData( sig_1, buffer, offset ) ;
 //offset = writeData( sig_0, buffer, offset ) ;

 //
 signal.trigger() ;
}

void keyPressed()
{
 // any keypress: toggle signal on / off
 signalOn = ! signalOn ;

 if( signalOn )
 {
 // reset the values to default when the signal turns back on
 cThrottle = DEFAULT_THROTTLE ;
 cPitch = DEFAULT_PITCH ;
 cYaw = DEFAULT_YAW ;
 cTrim = DEFAULT_TRIM ;
 programStartTime = millis() ;
 programIndex_ch1 = 0 ;
 println( "program started" ) ;
 }
 else
 {
 println( "program stopped" ) ;
 }
}

int writeValue( int numBits, int value, float[] target, int offset )
{
 byte tByte = (byte)value ;

 // nudge extra bits off
 int mask = 0x80;
 int bitDiff = 8 - numBits ;
 for( int i = 0 ; i < bitDiff ; i++ ) mask >>= 1;

 while( mask > 0 )
 {
 if( (mask & tByte) != 0)
 {
 offset = writeData( sig_1, target, offset ) ;
 //println( "wrote 1" ) ;
 }
 else
 {
 offset = writeData( sig_0, target, offset ) ;
 //println( "wrote 0" ) ;
 }
 mask >>= 1;
 }
 //println( ".." ) ;
 return offset ;
}

int writeData( float[] src, float[] target, int offset )
{
 int i ;
 for( i = 0 ; i < src.length ; i++ ) target[offset + i] = src[i] ;
 return offset + i ;
}

void stop()
{
 minim.stop();
 signal.close() ;

 super.stop();
}

class Instruction
{
 float time_s ;
 float throttle ;
 int pitch ;
 int yaw ;
 int trimOffset ;
 int channel ;
 int sprog ;

 Instruction( float time_s, float throttle, int pitch, int yaw, int trimOffset, int channel, int sprog )
 {
 this.time_s = time_s ;
 this.throttle = throttle ;
 this.pitch = pitch ;
 this.yaw = yaw ;
 this.trimOffset = trimOffset ;
 this.channel = channel ;
 this.sprog = sprog ;
 }
}

Troubleshooting:

I’ve tried this code on both Mac and Windows machines – for some reason that I don’t understand, I needed to invert the sign of the signal data on MacOS (see comment in code, within setup function) for the thing to work.

I note that the Appcopter app sets the output volume to max when the transmitter is plugged into the iPhone/iPad. Similarly, you need to set an alarmingly high volume for the transmitter to push out any infrared! Needs an amplifier in there methinks.

Sprog:

In the last post I indicated that the last two bits were some unknown signal element. After further study, this appears to be a control for the ‘headlight’ LED on the nose of the Appcopter. I suspect there may be more to it however – perhaps a ‘reset’ signal? I base this on the discovery that if the signal doesn’t go from 0 to 2 to 1 during a flight, the helicopter can be landed in an inconsistent state within which it’ll not respond to the control signal at all.

Flying time:

Ultimately, my ambition of commanding the helicopter to fly a square kind of worked… Check out the really poor video. I need a proper video camera.

Immediately you’ll note: It’s not a very square square. Uhh no.

Conclusions:

I quickly realised the naivety of my ambitions after the first couple of automated flights. The control signal was adequately deciphered, and the program fully sufficient to tell the helicopter what to do in a robotic way. My ‘flying turtle’ kind of worked, being able to draw a (very rough) square! But I expected perhaps a little more… accuracy?

Unlike the Turtle robot I once enjoyed at school, no two of my helicopter’s flights were the same. ‘Tiny’ variables, like rotor positioning at the start of a run, became massive deviations in direction as the flight progressed – reality brings the full weight of the chaos effect to bear on my toy robot! In my control signal I’ve tried to avoid a predictable episode of turbulent disruption caused by ground effect (the helicopter buffeted by its own rotor down wash) by launching up to a respectable altitude and stabilising before setting off. My flying robot with it’s four degrees of freedom (up/down, forwards/backwards, yaw, pitch) is a lot more complicated than a terrestrial buggy already at peace with gravity (and a mere two degrees of freedom)!

So what next? A sensible answer would be to attach sensors, and allow the helicopter to take care of it’s own stability to some extent. But my little toy can’t possibly be expected to carry a load (gyroscopic sensors, distance sensors, altimeter, the list of possible enhancements goes on), and with each new system comes a greater requirement for power, which obliges a bigger battery, which requires more lift, so bigger rotors or more powerful motors… Basically, my explorations with the Appcopter have led me to consider the possibilities of constructing a multirotor helicopter in order to explore ‘real’ onboard flight AI. I’m currently working on a hexcopter built from three broken Appcopters, in order to get a feel for constructing my own flying machine:

Advertisements
Tagged , , , ,

AppHack-e :: ‘Appcopter’ toy helicopter, hacked. Part I: The control signal

Motivation

A few weeks ago, I went to a local hobby and model shop to look for inspiration (and maybe something to dismantle). Of particular interest was this:

Appcopter

A small infra-red controlled toy helicopter, with a transmitter that plugs into an iPhone – simply download the virtual control panel from the app store to fly it. I was immediately struck by the potential for tinkering; it’s rare to see anything ‘unofficial’ plugged into an iPhone because the platform is so closed (Apple don’t let just anyone mess with their hardware), but here was a cheap toy making innovative use of the audio output socket, to fly a vehicle no less – that’s pretty clever! As a programmer I could see that in order to make a custom controller for this toy, all that’d be needed would be some code to ‘make the right noise down the wire’ to the transmitter, so to speak, so with this simple view in mind I went ahead and bought it with some idea of what I wanted to achieve…

I remember once playing with a programmable robot at school called ‘Logo’ (nicknamed ‘Turtle’), it was a white plastic dome shaped vehicle with wheels, that you could command to perform a series of navigations in sequence, leading the robot to traverse a maze or draw a picture. Program instructions were really simple, you could draw a square by commanding something like the following:

// draw a square: go forwards X units and turn right 90 degrees. Repeat four times.
Forward 10 ;
Right 90 ;
Forward 10 ;
Right 90 ;
Forward 10 ;
Right 90 ;
Forward 10 ;
Right 90 ;

I thought it’d be an interesting challenge to hack this toy helicopter into behaving like a ‘flying turtle’, so I could send it a series of instructions and watch as it performed them in sequence. That sounds like a pretty simple goal, but to get this far I’d have to achieve a number of tasks, most definitely learning a lot in the process. This modest goal also represents a necessary step on the path to bigger things, such as building a fully autonomous flight platform with sensory feedback systems – now that’d be awesome…

Control mechanics

So after playing with the helicopter for a bit to get a feel for the flight characteristics (cats now run from the room when the toy starts up), I looked at more detail into the available control mechanics.

The virtual controller app in d-pad mode

The virtual control panel gives you a number of controls, the most obvious being throttle – how fast the rotors spin, which translates into how quickly the helicopter ascends or descends. Get things balanced just right, and it hovers nicely.

A significant feature of this helicopter is that it has two pairs of ‘coaxially mounted’ rotor blades – each pair spins in opposite directions, with the effect of mutually canceling any bodily rotation induced during spin. The control circuitry on-board the helicopter manipulates the Y-axis rotation (yaw) of the helicopter in response to your command by slowing down or speeding up just one or the other pair of blades, deliberately allowing a little inductive torque in either direction. There’s also ‘trim‘ – it’s possible that the unique physiology of the helicopter (factory differences in motor power, for instance) means that some rotational force needs constant compensation; trim allows you to apply some permanent offsetting yaw for this.

Next is pitch, managed by a tiny motor and upwards-facing propeller on the tail boom – spin the rotor one way to dip the tail and make the helicopter fly backwards, or spin it the other to raise the tail and fly the helicopter forwards. Leaving the control untouched allows the helicopter to straighten out and hover.

The app offers two control modes. First is a ‘d-pad’ (‘digital pad’) for simple ‘zero-or-maximum’ forwards, backwards, yaw left, yaw right and combinations of these. Secondly there’s a pretty cool mode that allows you to fly the helicopter with the accelerometer of your iPhone by tilting it left, right, forwards or backwards to affect yaw and pitch respectively.

Finally, hidden away in another screen of the app, is a channel selection option: A, B or C – this setting lets you and some friends fly up to three helicopters on different channels, in the same room, each with their own transmitters and controller apps.

Control signal

When you fire up the throttle on the app, and the helicopter’s motors whirr into action, you can hear the plug-in transmitter box coughing out a staccato series of sounds through a tiny built-in speaker (which on first appraisal could passably be described as ‘helicopter rotor noises’, made for effect). The same sound is made through the speakers on the iPhone when the transmitter is unplugged, so there must be more to these sounds than mere aural aesthetic; it must be the control signal! It’s a pretty straightforward exercise to plug the iPhone’s audio output into the audio line-in port on a computer, and record a bit of the sound. It’s even easier to then look at it visually portrayed in a sound editing program – here’s what the audio signal looks like at two levels of zoom:

1) A little over 3.5 seconds of audio signal from the app, throttle set at 10%

2) Zoomed in upon one of the many individual ‘blips’ from the signal in image (1). The portion containing dips and peaks is about 30 milliseconds long.

Zoomed right into one of the many pulses visible in image (1), you can see that the signal looks like a series of valleys carved out from a line centered around zero (image 2). Though the signal looks like it tends upwards a bit across its entire length, the ‘valley floors’ along it look to be of pretty consistent widths and depths, with some looking like exactly double the width of the shorter ones.

This signal above looks most definitely digital (an analogue signal would present a more chaotic and varied looking series of smooth valleys and peaks or different heights and depths, not the regular ‘rectangular wave’ that we see). A little more research reveals that your average TV remote sends a signal that doesn’t look too different to the one sent by the helicopter transmitter… The important point being that the typical TV remote controller signal has a definite structure: it first opens with a ‘low signal’ or valley of a fixed length (like shouting someone’s name to get their attention), the signal data then follows like a spoken sentence, flowing in a series of words: control values spelled out in binary (with the length of the low signal denoting either a ‘1’ or a ‘0’), before closing with another fixed pattern of lows. Just in case the TV failed to get the message the first time around, the signal is then repeated. It looks like my helicopter similarly has a consistent ‘hello’ valley at the start (longest trough, far left of image 2), some binary data then follows (with what could be a longer valley denoting a ‘1’, and a shorter valley denoting ‘0’) before coming to a definitive close. The entire signal is repeated again over and over at regular intervals so long as the throttle is greater than zero – setting the throttle to zero stops the signal audio.

A brief moment to explain binary…

When we count from 0 to 9, we visit a series of unique numerical symbols in one ‘column’, which we call the ‘units‘: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9. When we get to the number 10, we wrap over into two columns: the ‘tens‘ (of which we have one in this instance) in addition to the ‘units‘ (of which we have none). Count up to 100 and we have one ‘hundred‘, no ‘tens‘ and no ‘units‘. Following so far? We’ve just been counting in ‘base-10’ – our numerical system is in fact built around base-10. We don’t have to constrain ourselves to ‘units‘, ‘tens‘, ‘hundreds‘, etc. though – you can effectively work in a numerical system of any base – binary is simply counting in ‘base-2’! The table below shows exactly what that looks like – on the left is the familiar number in base-10, on the right is the same number, in binary. See that instead of ‘tens‘, ‘hundreds‘ and ‘thousands‘ (powers of ten), we instead have ‘twos‘, ‘fours‘, ‘eights‘, ‘sixteens‘, etc. (powers of two), with each column being shifted along to the left when the smallest gets filled.

1 (one ‘unit’) → 1 (one ‘one’)

2 (two ‘units’) → 10 (no ‘ones’ plus one ‘two’)

3 (three ‘units’) → 11 (one ‘one’ plus one ‘two’)

4 (four ‘units’) → 100 (one ‘four’ plus no ‘two’ and no ‘one’)

5 (five ‘units’) → 101 (one ‘four’ plus no ‘two’ and one ‘one’)

6 (six ‘units’) → 110 (one ‘four’ plus one ‘two’ and no ‘one’)

7 (seven ‘units’) → 111 (one ‘four’ plus one ‘two’ and one ‘one’)

8 (eight ‘units’) → 1000 (one ‘eight’ plus no ‘fours’ plus no ‘two’ and no ‘one’)

9 (nine ‘units’) → 1001 (one ‘eight’ plus no ‘fours’ plus no ‘two’ and one ‘one’)

10 (one ‘ten’, no ‘units’)→ 1010 (one ‘eight’ plus no ‘fours’ plus one ‘two’ and no ‘one’)

When using binary in computing, it’s typical to specify a number with a certain length of symbols – like putting in a placeholder ‘0’ for each column we might possibly use – this is the number of ‘bits’ (‘binary digits’) used to specify the number:

The number 1 in ‘four bit binary’: 0001

The number 15 in ‘four bit binary’: 1111

So what happens if we try to specify the number 16 in four bits? We just can’t do it! There’s a limit to the maximum number we can specify with a certain number of bits. To show the number 16 in binary we need at least five bits:

The number 16 in ‘five bit binary’: 10000

Decoding the binary signal

So we’ve seen what controls the helicopter responds to: throttle, yaw, pitch, trim, and channel. The signal data must contain information about each of these – there’s no magic happening when you push up the throttle control! We’ve established that the signal is binary in format; it’s sending a series of numbers represented by zeroes and ones – these numbers must mean something about each control, but in what order do the values get sent, and by how many bits is each represented? With applied scientific method, we can easily figure out the signal encoding! The process is really simple:

– Choose just one variable (be it throttle, yaw, pitch, trim or channel) and modify it to a known value, for example, set throttle at 10%. Record the signal audio – this is our ‘control’ signal, against which all further modifications will be compared.

– Change the selected variable by a small amount, for example, set throttle to 20%. Record the signal audio.

– Open up the audio file for the control signal (in this case throttle at 10%), and place alongside it the audio for the latest recording (throttle at 20%) – see how the signal changes between the two wave forms (image 3) – there’s clearly a connection between a particular variable (throttle in this case) and the part of the second recording that changes in response.

– Repeat many times, for a range of control settings for a particular variable, before moving onto a different variable and starting over!

3) Signal comparison: Throttle at 10% (top) and 100% (bottom). There’s definitely something going on with those first few bits of signal (highlit, bottom)

I found it possible to get a pretty big hint about the location of a particular control value on the signal, with a little trick: the controls have maximum and minimum values (for example, throttle has both a 0% and 100% setting), so assuming these aren’t mapped to weird arbitrary values, they must be equivalent to the value on the signal – the signal should show the highest value for throttle, say, when the app has it set at 100%. So set the throttle to 100%, and the signal shows a series of long valleys (obviously then our ‘ones’) at a location that must contain the throttle value!

I initially started writing a program to automate the full testing procedure, exporting the audio files to a text file format (easier to crunch the numbers that way). I eventually ran out of patience with this though after encountering a number of issues, and resorted to pencil and paper – it turned out to be much quicker and easier – about an hour to figure out the encoding, and like any puzzle, really good fun.

Pencil and paper: Change one aspect of the control (e.g. set throttle @ 20%) and see how the signal changes relative to some base setting (I used throttle @ 10% as my reference signal). Repeat many times, identifying the characteristics of the control components of the signal

The signal exposed

So here’s a summary of my findings, and everything you need to start writing a program to take control of the helicopter:

– The audio signal is a rectangular waveform representing binary data, with a normalised peak amplitude ‘pulse’ of about 0.6 units, when represented in uncompressed .wav file format.

– The signal begins with an ‘on’ pulse of ~1600 microseconds

– A binary ‘0’ is indicated by a pulse of ~400 microseconds following a gap of ~400 microseconds

– A binary ‘1’ is indicated by a pulse of ~800 microseconds following a gap of ~400 microseconds

– The entire signal carries 28 bits

– Following the on pulse the controls are sent as follows:

Throttle: 7-bits, defaulting to 0, full at 127
Pitch: 6-bits, defaulting to 32, full forwards at 0, full reverse at 63
Yaw: 5-bits, defaulting to 16, full right at 0, full left at 31
Trim: 6-bits, defaulting to 31, full right at 0, full left at 63
Channel: 2-bits, A: 1, B: 2, C: 0
???: 2-bits, no idea – let’s find out in the next post!

That’s it for now – there’s a lot of content here, so I’ll deliver the story in two parts. In the next post I’ll reveal the program written in Processing to generate the audio signal for the transmitter, and talk about automating the helicopter’s flight!

Tagged , , , , , ,