Halloween Project (Garage Of Mystery)

November 4, 2014 2 Comments

For this years Halloween, the kids and I decided to do something out of the opening scene of Indiana Jones, without the big rock.  We wanted to give kids a choice when they came to the house –> either get a small “fun” size candy bar or enter the garage of mystery for the chance of a full sized candy bar.  (Incidentally, whoever thought it would be a good idea to name the smallest candy size on earth “fun” obviously was never a kid.  When I was growing up, we called it four size, being that if took four of them to make a normal candy bar)

So if the kid wants to go into the garage of mystery, they have to get to the alter of snickers without the motion detector or the laser beam trip wires catching them.  The full-size Snickers would disappear if the kid was picked up by the Kinect motion detector or if they tripped too many beams.  In the diagram below, the red dots are the lasers crossing in front of the alter

image

The first thing we did was construct the alter.  

imageimage

Once the frame was set, we added a servo with a trap door to the top.  We control the servo via a Phidget Servo Controller with some basic code from the Phidget SDK (if the SDK, you know, had F# in it)

 1     member this.servoController_Attached(args:Events.AttachEventArgs) =
 2         let _servoController = args.Device :?> AdvancedServo
 3         _servoController.servos.[0].Engaged <- true 
 4         _servoController.servos.[0].Position <- 110. 
 5         _isServoControllerReady  <- true
 6 
 7     member this.initializeController() =
 8         _servoController.Attach.Add(this.servoController_Attached)
 9         _servoController.``open``()
10 
11     member this.moveController(position:float) =
12         if _isServoControllerReady then
13             _servoController.servos.[0].Position <- position 
14

And you can see it in action here:

 

With the alter ready, we turned our attention to the laser trip wires.  We purchased a whole bunch of dollar store pen lasers and got some Phidget light sensors.  We then created a frame for both sides of the garage –> one to mount the laser and 1 to mount the light sensor

imageimage

And then we added some basic code from the Phidget SDK (if the SDK, you know, had F# in it)

 1     member this.interfaceKit_Attached(args: Events.AttachEventArgs) =
 2         let _interfaceKit = args.Device :?> InterfaceKit
 3         _interfaceKit.sensors 
 4                         |> Seq.cast
 5                         |> Seq.map(fun s -> s :> InterfaceKitAnalogSensor)
 6                         |> Seq.map(fun s -> s.Sensitivity <- 20)
 7                         |>ignore
 8         _isInterfaceKitReady <- true
 9 
10     member this.interfaceKit_SensorChange(e: SensorChangeEventArgs ) =
11         let eventArgs = new LightSensorChangeEventArgs(e.Index,e.Value)
12         lightSensorChange.Trigger(eventArgs)
13 
14     member this.initializeInterfaceKit() =
15         _interfaceKit.Attach.Add(this.interfaceKit_Attached)
16         _interfaceKit.SensorChange.Add(this.interfaceKit_SensorChange)
17         _interfaceKit.``open``()
18         _interfaceKit.waitForAttachment()
19

Note that we are trapping the event from the light sensor and then raising it up in our own event. 

With the light sensor in place, we turned our attention to the Kinect motion sensor.  I first considered Rob Miles’s ides to compare the different color frames to see if there was movement but because I am using F# and F# does not support pointers like C#, the performance was too choppy.  You can see the Stack Overflow thread here.  So I could have either jumped to over to C# or figure out a different way using F#.  I went with option B by using the skeleton frame, which has a Z index.  By comparing the Z index over time, I can see how fast a person is moving towards to alter.  The Kinect code was pretty much from the SDK (if the SDK, you know, had F# in it)

 1     member this.kinectSensor_ColorFrameReady(args: ColorImageFrameReadyEventArgs) =
 2         use colorFrame = args.OpenColorImageFrame()
 3         if not (colorFrame = null) then
 4             let colorData =  Array.zeroCreate<byte> colorFrame.PixelDataLength 
 5             colorFrame.CopyPixelDataTo(colorData)
 6             let width = colorFrame.Width
 7             let height = colorFrame.Height
 8             let stride = colorFrame.Width * colorFrame.BytesPerPixel
 9             let eventArgs = new ColorDataReadyEventArgs(colorData,width,height,stride)
10             colorDataReady.Trigger(eventArgs)
11             ()
12 
13     member this.KinectSensor_SkeletonFrameReady(args: SkeletonFrameReadyEventArgs) =
14         use skeletonFrame = args.OpenSkeletonFrame()
15         if not (skeletonFrame = null) then
16             let skeletons = Array.zeroCreate<Skeleton> skeletonFrame.SkeletonArrayLength
17             skeletonFrame.CopySkeletonDataTo(skeletons)
18             let skeletons1 = skeletons |> Array.filter (fun s -> s.TrackingState = SkeletonTrackingState.Tracked)               
19             if skeletons1.Length > 0 then
20               skeletonChanged.Trigger(skeletons1.[0])
21             ()
22         ()
23 
24     member this.initializeKinect() =
25         _kinectSensor.ColorStream.Enable()
26         _kinectSensor.ColorFrameReady.Subscribe(this.kinectSensor_ColorFrameReady) |> ignore
27         _kinectSensor.SkeletonStream.Enable();
28         _kinectSensor.SkeletonFrameReady.Subscribe(this.KinectSensor_SkeletonFrameReady) |> ignore
29         _kinectSensor.Start()
30

In the UI, I then checked for the skeleton movement and if the person moved too fast, they would trigger the snickers trap door to open

 1         void garage_SkeletonChanged(object sender, Skeleton skeleton)
 2         {
 3             if(_skeletonPoint.Z > 0)
 4             {
 5                 float zDelta = _skeletonPoint.Z - skeleton.Position.Z;
 6                 if (zDelta >= _zDeltaThreshold)
 7                 {
 8                     _numberOfSkeletonHits += 1;
 9                     skeletonChangedProgressBar.Dispatcher.Invoke(new Action(() => skeletonChangedProgressBar.Value = _numberOfSkeletonHits));
10 
11                 }
12                 if(_numberOfSkeletonHits >= _numberOfHitsForAlarm)
13                 {
14                     _garage.moveController(_openPosition);
15                 }
16 
17                 skeletonCanvas.Children.Clear();
18                 drawSkelton(skeleton);
19             }
20             _skeletonPoint = skeleton.Position;
21         }
22

With the result like this:

With the hard parts done, it was time to create a UI.  I went with C# here because I am using WPF and the support for WPF and the Kinect is best in C#.  I created a WPF application and built a UI

 1 <Window x:Class="ChickenSoftware.Halloween.UI.MainWindow"
 2         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 3         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 4         Title="MainWindow" Height="600" Width="650" >
 5     <Grid Height="600" Width="650" VerticalAlignment="Top" HorizontalAlignment="Left" >
 6         <Image Name ="kinectVideo"  Height="480" Width="640" Margin="10,0,0,120" />
 7         <Canvas Name="skeletonCanvas" Height="480" Width="640" Margin="10,0,0,120"  />
 8         <Rectangle x:Name="sensor0Rectange" Fill="Lime" HorizontalAlignment="Left" Height="40" Margin="10,480,0,0" Stroke="Black" VerticalAlignment="Top" Width="82"/>
 9         <Rectangle x:Name="sensor1Rectange" Fill="Lime" HorizontalAlignment="Left" Height="40" Margin="189,480,0,0" Stroke="Black" VerticalAlignment="Top" Width="83"/>
10         <Rectangle x:Name="sensor2Rectange" Fill="Lime" HorizontalAlignment="Left" Height="40" Margin="367,480,0,0" Stroke="Black" VerticalAlignment="Top" Width="82"/>
11         <Rectangle x:Name="sensor3Rectange" Fill="Lime" HorizontalAlignment="Left" Height="40" Margin="537,480,0,0" Stroke="Black" VerticalAlignment="Top" Width="83"/>
12         <ProgressBar x:Name="skeletonChangedProgressBar" HorizontalAlignment="Left" Height="40" Margin="10,528,0,0" VerticalAlignment="Top" Width="392" Foreground="#FFB00606"/>
13         <Button x:Name="resetButton" Content="Reset" HorizontalAlignment="Left" Height="37" Margin="537,528,0,0" 
14                 VerticalAlignment="Top" Width="83" Click="resetButton_Click"/>
15         <Button x:Name="EjectButton" Content="Eject!" HorizontalAlignment="Left" Height="37" Margin="429,528,0,0" 
16             VerticalAlignment="Top" Width="83" Click="EjectButton_Click"/>
17     </Grid>
18 </Window>

I then added some code to handle all of the events that the Phidgets and Kinect are sending to the UI and do something useful with it.    For example, the light sensor change fills in the appropriate box on the screen (note that Phidgets use a different thread so you need to use Dispatcher.Invoke)

 1         void garage_LightSensorChange(object sender, LightSensorChangeEventArgs args)
 2         {
 3             switch (args.SensorIndex)
 4             {
 5                 case 0:
 6                     if (args.SensorIndex == 0 && args.LightAmount < _lightSensorThreshold)
 7                     {
 8                         _sensor0Tripped = true;
 9                         sensor0Rectange.Dispatcher.Invoke(new Action(()=>sensor0Rectange.Fill = new SolidColorBrush(Colors.Red)));
10                     }
11                     break;
12                 case 1:
13 
14                     if (args.SensorIndex == 1 && args.LightAmount < _lightSensorThreshold)
15                     {
16                         _sensor1Tripped = true;
17                         sensor1Rectange.Dispatcher.Invoke(new Action(() => sensor1Rectange.Fill = new SolidColorBrush(Colors.Red)));
18                     }
19                     break;
20                 case 2:
21                     if (args.SensorIndex == 2 && args.LightAmount < _lightSensorThreshold)
22                     {
23                         _sensor2Tripped = true;
24                         sensor2Rectange.Dispatcher.Invoke(new Action(() => sensor2Rectange.Fill = new SolidColorBrush(Colors.Red)));
25                     }
26                     break;
27                 case 3:
28                     if (args.SensorIndex == 3 && args.LightAmount < _lightSensorThreshold)
29                     {
30                         _sensor3Tripped = true;
31                         sensor3Rectange.Dispatcher.Invoke(new Action(() => sensor3Rectange.Fill = new SolidColorBrush(Colors.Red)));
32                     }
33                     break;
34             }
35             CheckForIntruder();
36         }
37

With this associated method 

 1         private void CheckForIntruder()
 2         {
 3             Int32 numberOfSensorsTripped = 0;
 4 
 5             if (_sensor0Tripped == true) 
 6                 numberOfSensorsTripped += 1;
 7             if (_sensor1Tripped == true)
 8                 numberOfSensorsTripped += 1;
 9             if (_sensor2Tripped == true)
10                 numberOfSensorsTripped += 1;
11             if (_sensor3Tripped == true)
12                 numberOfSensorsTripped += 1;
13             if (numberOfSensorsTripped >= _numberOfSensorsForAlarm )
14                 _garage.moveController(0);
15 
16         }

This code would be so much better in F# using pattern matching but b/c of the UI code, I kept it in C#.  I might refactor the non-visual components later.  The one thing that did surprise me is that how the Kinect V1 SDK makes it very hard to separate the UI components from the domain components.  Phidgets, on the other hand, had a very clear separation of concerns

So we then added some sides to the alter of snickers

image

And we were good to go.  The final result looks like this (the smoke machine was an added touch):

All of the code is on github here.  If you create your own garage of mystery, please drop me a line –> I would love to see what other makers come up with. 

Filed under F#, Kinect, Phidgets

Controlling Servos Using Netdunio and Phidgets

August 12, 2014 1 Comment

As part of the Terminator program I am creating, I need a way of controlling servos to point the laser (and then gun) and different targets.  I decided to create a POC project and evaluate two different ways of controlling the servos.  As step one, I purchased a pan and tilt chassis from here. 

image

After playing with the servos from the kit, I decided to use my old stand-by servos that had a much higher quality and whose PWM signals I already know how to use.  With the chassis done, I needed a laser pointer so I figured why not get a shark with fricken laser?

I found one here.

image

So with the servos and laser ready to go, it was time to code.  I started with Netduninos:

    public class Program
    {
        private const uint TILT_SERVO_STRAIGHT = 1500;
        private const uint TILT_SERVO_MAX_UP = 2000;
        private const uint TILT_SERVO_MAX_DOWN = 1000;
        private const uint PAN_SERVO_STRAIGHT = 1500;
        private const uint PAN_SERVO_MAX_LEFT = 1000;
        private const uint PAN_SERVO_MAX_RIGHT = 2000;

        private static PWM _tiltServo = null;
        private static PWM _panServo = null;
        private static uint _tiltServoCurrentPosition = 0;
        private static uint _panServoCurrentPosition = 0;

        public static void Main()
        {
            SetUpServos();
            InputPort button = new InputPort(Pins.ONBOARD_BTN, false, Port.ResistorMode.Disabled);
            while (true)
            {
                if (button.Read())
                {
                    MoveServo();
                }
            }

        }

        private static void SetUpServos()
        {
            uint period = 20000;
            _tiltServoCurrentPosition = TILT_SERVO_STRAIGHT;
            _panServoCurrentPosition = PAN_SERVO_STRAIGHT;

            _tiltServo = new PWM(PWMChannels.PWM_PIN_D3, period, _tiltServoCurrentPosition, PWM.ScaleFactor.Microseconds, false);
            _tiltServo.Start();
            _panServo = new PWM(PWMChannels.PWM_PIN_D5, period, _panServoCurrentPosition, PWM.ScaleFactor.Microseconds, false);
            _panServo.Start();
        }

        private static void MoveServo()
        {
            _panServo.Duration = PAN_SERVO_MAX_LEFT;
            Thread.Sleep(2000);
            _panServo.Duration = PAN_SERVO_MAX_RIGHT;
            Thread.Sleep(2000);
            _panServo.Duration = PAN_SERVO_STRAIGHT;
            Thread.Sleep(2000);
            _tiltServo.Duration = TILT_SERVO_MAX_UP;
            Thread.Sleep(2000);
            _tiltServo.Duration = TILT_SERVO_MAX_DOWN;
            Thread.Sleep(2000);
            _tiltServo.Duration = TILT_SERVO_STRAIGHT;
        }

    }

And sure enough the servos are behaving as expected

I then implemented a similar app using Phidgets.  Because the code is being executed on the PC, I could use F# to code (It does not look like the Netdunino/Microframework supports F#?)

open System
open Phidgets

let _servoController = new AdvancedServo()
let mutable _isServoControllerReady = false

let servoController_Attached(args:Events.AttachEventArgs) =
    let servoController =  args.Device  :?> AdvancedServo
    servoController.servos.[0].Engaged <- true
    servoController.servos.[7].Engaged <- true
    _isServoControllerReady <- true

[<EntryPoint>]
let main argv = 
    _servoController.Attach.Add(servoController_Attached)
    _servoController.``open``()

    while true do
        if _isServoControllerReady = true then
            _servoController.servos.[0].Position<- 100.
            _servoController.servos.[7].Position<- 100.

    Console.ReadKey() |> ignore
    printfn "%A" argv
    0

 

The choice then becomes using the Netduino or the Phidgets with my Kinect program.  I decided to defer the decision and use an interface for now.

type IWeaponsSystem =
    abstract member Activate: unit -> unit
    abstract member AquireTarget : float*float -> bool
    abstract member Fire: int -> bool

My decision about using Phidgets or Netduino is a series of trade-offs.  I can code Phidgets in C# or F# but I have to code Netduino in C#.  I would prefer to do this in F# so that makes me learn towards Phidgets.  I can put the Netduino anywhere and have it communicate via an Ethernet signal but I have to have the Phidgets wired to the PC.  Since the targeting system needs to be near the Kinect and the Kinect has to be tethered to the PC also, there is no real advantage of using the mobile Netduino.  Finally, the Phidgets API handles all communication to the servo control board for me, with the Netduino I would have to hook up a router to the Netduino and write the Ethernet communication code.  So I am leaning towards Phidgets, but since I am not sure, the interface allows me to swap in the Netduino at a later point without changing any code.  Love me some O in SOLID…

Up next, integrating the targeting system into the Terminator program.

 

 

Filed under F#, Netduino, Phidgets