Terminator Program: With The Kinect 2

July 29, 2014 Leave a comment

I got my hands on a Kinect2 last week so I decided to re-write the Terminator program using the Kinect2 api.  Microsoft made some major changes to the domain api (no more skeleton frame, now using a body) but the underlying logic is still the same.  Therefore, it was reasonably easy to port the code.  There is plenty of places in the V2 api that are not documented yet but because I did some work in the V1 api, I could still get things done.  For example, the V2 api documentation and code samples use event handlers to work with any new frame that arrives from the Kinect.  This lead to some pretty laggy code.  However, by using polling on a second thread, I was able to get the performance to where it needs to be.  Also, a minor annoyance is that you have to use Win8 with the Kinect 2.

So here is the Terminator application, Gen 2.  The UI is still just a series of UI controls:

 1 <Window x:Class="ChickenSoftware.Terminator.Gen2.UI.MainWindow"
 2         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 3         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 4         Title="MainWindow" Height="700" Width="650" Loaded="Window_Loaded">
 5     <Canvas Width="650" Height="700">
 6         <Image x:Name="kinectColorImage" Width="640" Height="480" />
 7         <Canvas x:Name="bodyCanvas" Width="640" Height="480" />
 8         <Button x:Name="takePhotoButton" Canvas.Left="10" 
 9                 Canvas.Top="485" Height="40" Width="125" Click="takePhotoButton_Click">Take Photo</Button>
10         <TextBox x:Name="facialRecognitionTextBox" Canvas.Left="10" Canvas.Top="540" Width="125" Height="40" FontSize="8" />
11         <Image x:Name="currentImage" Canvas.Left="165" Canvas.Top="485" Height="120" Width="170" />
12         <Image x:Name="compareImage" Canvas.Left="410" Canvas.Top="485" Height="120" Width="170" />
13     </Canvas>
14 </Window>
15

In the code behind, I set up some class-level variables.  The only real difference is that the photo is moving from 640/480 to 1920/1080:

1         KinectSensor _kinectSensor = null;
2         Boolean _isKinectDisplayActive = false;
3         Boolean _isTakingPicture = false;
4         WriteableBitmap _videoBitmap = null;
5         Int32 _width = 1920;
6         Int32 _height = 1080;

When the page is loaded, a new thread is spun up that handles rendering the Kinect data:

1         private void Window_Loaded(object sender, RoutedEventArgs e)
2         {
3             SetUpKinect();
4             _isKinectDisplayActive = true;
5             Thread videoThread = new Thread(new ThreadStart(DisplayKinectData));
6             videoThread.Start();
7         }

Setting up the Kinect is a bit different (KinectSensor.GetDefault()) but intuitive:

1         internal void SetUpKinect()
2         {
3             _videoBitmap = new WriteableBitmap(1920, 1080, 96, 96, PixelFormats.Bgr32, null);
4             _kinectSensor = KinectSensor.GetDefault();
5             _kinectSensor.Open();
6         }

With the big change in the DisplayKinectData method

 1         internal void DisplayKinectData()
 2         {
 3             var colorFrameSource = _kinectSensor.ColorFrameSource;
 4             var colorFrameReader = colorFrameSource.OpenReader();
 5             var bodyFrameSource = _kinectSensor.BodyFrameSource;
 6             var bodyFrameReader = bodyFrameSource.OpenReader();
 7 
 8             while (_isKinectDisplayActive)
 9             {
10                 using (var colorFrame = colorFrameReader.AcquireLatestFrame())
11                 {
12                     if (colorFrame == null) continue;
13                     using (var bodyFrame = bodyFrameReader.AcquireLatestFrame())
14                     {
15                         if (bodyFrame == null) continue;
16                         //Color
17                         var colorFrameDescription = colorFrame.ColorFrameSource.CreateFrameDescription(ColorImageFormat.Bgra);
18                         var bytesPerPixel = colorFrameDescription.BytesPerPixel;
19                         var frameSize = colorFrameDescription.Width * colorFrameDescription.Height * bytesPerPixel;
20                         var colorData = new byte[frameSize];
21                         if (colorFrame.RawColorImageFormat == ColorImageFormat.Bgra)
22                         {
23                             colorFrame.CopyRawFrameDataToArray(colorData);
24                         }
25                         else
26                         {
27                             colorFrame.CopyConvertedFrameDataToArray(colorData, ColorImageFormat.Bgra);
28                         }
29                         //Body
30                         var bodies = new Body[bodyFrame.BodyCount];
31                         bodyFrame.GetAndRefreshBodyData(bodies);
32                         var trackedBody = bodies.FirstOrDefault(b => b.IsTracked);
33 
34                         //Update
35                         if (_isTakingPicture)
36                         {
37                             Dispatcher.Invoke(new Action(() => AnalyzePhoto(colorData)));
38                         }
39                         else
40                         {
41                             if (trackedBody == null)
42                             {
43                                 Dispatcher.Invoke(new Action(() => UpdateDisplay(colorData)));
44                             }
45                             else
46                             {
47                                 Dispatcher.Invoke(new Action(() => UpdateDisplay(colorData, trackedBody)));
48                             }
49                         }
50                     }
51                 }
52             }
53         }
54

I am using a frameReader and frameSource for both the color (the video image) and the body (the old skeleton).  The method to get the frame has changed –> I am using AquireLatestFrame().  It is nice that we are still using byte[] to hold the data.

With the data in the byte[] arrays, the display is updated.  There are two UpdateDisplay methods:

 1         internal void UpdateDisplay(byte[] colorData)
 2         {
 3             var rectangle = new Int32Rect(0, 0, _width, _height);
 4             _videoBitmap.WritePixels(rectangle, colorData, _width * 4, 0);
 5             kinectColorImage.Source = _videoBitmap;
 6         }
 7 
 8         internal void UpdateDisplay(byte[] colorData, Body body)
 9         {
10             UpdateDisplay(colorData);
11             var drawingGroup = new DrawingGroup();
12             using (var drawingContext = drawingGroup.Open())
13             {
14                 var headPosition = body.Joints[JointType.Head].Position;
15                 if (headPosition.Z < 0)
16                 {
17                     headPosition.Z = 0.1f;
18                 }
19                 var adjustedHeadPosition = _kinectSensor.CoordinateMapper.MapCameraPointToDepthSpace(headPosition);
20                 bodyCanvas.Children.Clear();
21                 Rectangle headTarget = new Rectangle();
22                 headTarget.Fill = new SolidColorBrush(Colors.Red);
23                 headTarget.Width = 10;
24                 headTarget.Height = 10;
25                 Canvas.SetLeft(headTarget, adjustedHeadPosition.X + 75);
26                 Canvas.SetTop(headTarget, adjustedHeadPosition.Y);
27                 bodyCanvas.Children.Add(headTarget);
28             }
29         }

This is pretty much like V1 where the video byte[] is being written to a WritableBitmap and the body is being drawn on the canvas.  Note that like V1, the coordinates of the body need to be adjusted to the color frame.  The API has a series of overloads that makes it easy to do the translation.

With the display working, I added in taking the photo, sending it to Azure blob storage, and having Sky Biometry analyze the results.  This code is identical to V1 with the connection strings for Azure and Sky Biometry broken out into their own methods and the sensitive values placed into the app.config:

1         internal void AnalyzePhoto(byte[] colorData)
2         {
3             var bitmapSource = BitmapSource.Create(_width, _height, 96, 96, PixelFormats.Bgr32, null, colorData, _width * 4);
4             JpegBitmapEncoder encoder = new JpegBitmapEncoder();
5             encoder.Frames.Add(BitmapFrame.Create(bitmapSource));
6             var photoImage = UploadPhotoImage(encoder);
7             CompareImages(photoImage);
8             _isTakingPicture = false;
9         }

 1         internal PhotoImage UploadPhotoImage(JpegBitmapEncoder encoder)
 2         {
 3             using(MemoryStream memoryStream = new MemoryStream())
 4             {
 5                 encoder.Save(memoryStream);
 6                 var photoImage = new PhotoImage(Guid.NewGuid(), memoryStream.ToArray());
 7 
 8                 var customerUniqueId = new Guid(ConfigurationManager.AppSettings["customerUniqueId"]);
 9                 var connectionString = GetAzureConnectionString();
10 
11                 IPhotoImageProvider provider = new AzureStoragePhotoImageProvider(customerUniqueId, connectionString);
12                 provider.InsertPhotoImage(photoImage);
13                 memoryStream.Close();
14                 return photoImage;
15             }
16         }

 1         internal void CompareImages(PhotoImage photoImage)
 2         {
 3             String skyBiometryUri = ConfigurationManager.AppSettings["skyBiometryUri"];
 4             String uid = ConfigurationManager.AppSettings["skyBiometryUid"];
 5             String apiKey = ConfigurationManager.AppSettings["skyBiometryApiKey"];
 6             String apiSecret = ConfigurationManager.AppSettings["skyBiometryApiSecret"];
 7             var imageComparer = new SkyBiometryImageComparer(skyBiometryUri, uid, apiKey, apiSecret);
 8 
 9             String basePhotoUri = GetBasePhotoUri();
10             String targetPhotoUri = GetTargetPhotoUri(photoImage);
11             currentImage.Source = new BitmapImage(new Uri(targetPhotoUri));
12             compareImage.Source = new BitmapImage(new Uri(basePhotoUri));
13 
14             var matchValue = imageComparer.CalculateFacialRecognitionConfidence(basePhotoUri, targetPhotoUri);
15             facialRecognitionTextBox.Text = "Match Value Confience is: " + matchValue.Confidence.ToString();
16         }

With the code in place, I can the run the Terminator Gen 2:

image

I think I am doing the Sky Biometry recognition incorrectly so I will look at that later.  In any event, working with the Kinect V2 was fairly easy because it was close enough to the V1 that the concepts could translate.  I look forward to adding the targeting system this weekend!!!

Filed under Kinect

Neural Network Part 2: Perceptrons

July 22, 2014 1 Comment

I started working though the second chapter of McCaffrey’s book Neural Networks Using C# Succinctly to see if I could write the examples using F#.

McCaffrey’s code is tough to read though because of its emphasis on loops and global mutable variables.  I read though his description and this is how <I think> the Perceptron should be constructed.

The inputs are a series of independent variables (in this case age and income) and the output is a single dependent variable (in this case party affiliation).  The values have been encoded and normalized like in this post here.

An example of the input (from page 31 of his book) is:

image

Or in a more abstract manner:

image

In terms of data structures, individual inputs (each row) is placed into an array of floats and the output is a single float

image

I call this single set of inputs an “observation” (my words, not McCaffrey).

Looking at McCaffrey’s example for a perceptron Input-Output,

image

all of the variables you need are not included. Here is what you need:

image

Where A0 and B0 are the same as X0 and X1 respectively in his diagram.  Also, McCaffrey uses the word “Perceptron” to mean two different concepts: the entire system as a whole and the individual calculation for a given list of X and Bias.  I am a big believer of domain ubiquitous languages so I am calling the individual calculation a neuron.

Once you run these values through the neuron for the 1st observation, you might have to alter the Weights and Bias based on the (Y)result.  Therefore, the data structure coming out of the Neuron is

image

These values are feed into the adjustment function to alter the weights and bias with the output as

image

I am calling this process of taking the a single observation, the xWeights, , and the bias and turning them into a series of weights and bais as a “cycle” (my words, not McCaffrey)

image

 

The output of a cycle is then fed with  the next observation and the cycle repeats for as many observations as there are fed into the system. 

image

 

I am calling the process of running a cycle for each observation in the input dataset a rotation (my words, not McCaffrey) and that the perceptron runs rotations for an x number of times to train itself.

image

 

Finally, the Perceptron takes a new set of observations where the Y is not known and runs a Rotation once to predict what the Y will be.

So with that mental image in place, the coding became much easier.  Basically, there was a 1 to 1 correspondence of F# functions to each step laid out.  I started with an individual cycle

  1. type cycleInput = {xValues:float list;yExpected:float;mutable weights:float list;mutable bias:float;alpha:float}
  2.  
  3. let runNeuron (input:cycleInput) =
  4.     let valuesAndWeights = input.xValues |> List.zip input.weights
  5.     let output = valuesAndWeights
  6.                     |> List.map(fun (xValue, xWeight) -> xValue*xWeight)
  7.                     |> List.sumBy(fun x -> x)
  8.     output + input.bias
  9.  
  10. let runActivation input =
  11.     if input < 0.0 then -1.0 else 1.0

I used record types all over the place in this code just so I could keep things straight in my head.  McCaffrey uses ambiguously-named arrays and global variables.  Although this makes my code a bit more wordy (esp for functional people), I think the increased readability is worth the trade-off.

In any event, with the Neuron and Activation calc out of the way, I created the functions that adjust the weights and bias:

  1. let calculateWeightAdjustment(xValue, xWeight, alpha, delta) =
  2.     match delta > 0.0, xValue >= 0.0 with
  3.         | true,true -> xWeight – (alpha * delta * xValue)
  4.         | false,true -> xWeight + (alpha * delta * xValue)
  5.         | true,false -> xWeight – (alpha * delta * xValue)
  6.         | false,false -> xWeight + (alpha * delta * xValue)
  7.  
  8. let calculateBiasAdjustment(bias, alpha, delta) =
  9.     match delta > 0.0 with
  10.         | true -> bias – (alpha * delta)
  11.         | false -> bias + (alpha * delta)

This code is significantly different than the for, nested if that McCaffrey uses. 

image

I maintain using this kind of pattern matching makes the intention much easier to comprehend.  I also split out the adjustment of the weights and the adjustment of the bias into individual functions.

With these functions ready, I created an input and output record type and implemented the adjustment function

  1. let runAdjustment (input:adjustmentInput) =
  2.     match input.yExpected = input.yActual with
  3.         | true -> {weights=input.weights;bias=input.bias;yActual=input.yActual}
  4.         | false ->
  5.             let delta = input.yActual – input.yExpected
  6.             let valuesAndWeights = input.xValues |> List.zip input.weights
  7.             let weights' =  valuesAndWeights |> List.map(fun (xValue, xWeight) -> calculateWeightAdjustment(xValue,xWeight,input.alpha,delta))
  8.             let bias' = calculateBiasAdjustment(input.bias,input.alpha,delta)
  9.             {weights=weights';bias=bias';yActual=input.yActual}

There is not a corresponding method in McCaffrey’s code, rather he just does some Array.copy and mutates the global variables in the Update method.  I am not a fan of side-effect programming so I created a function that explicitly does the  modification.

And to wrap up the individual cycle:

  1. let runCycle (cycleInput:cycleInput) =
  2.     let neuronResult = runNeuron(cycleInput)
  3.     let activationResult = runActivation(neuronResult)
  4.     let adjustmentInput = {xValues=cycleInput.xValues;weights=cycleInput.weights;yExpected=cycleInput.yExpected;
  5.                             bias=cycleInput.bias;alpha=cycleInput.alpha;yActual=activationResult}
  6.     runAdjustment(adjustmentInput)

Up next is to run the cycle for each of the observations (called a rotation)

  1. type observation = {xValues:float list;yExpected:float}
  2. type rotationInput = {observations: observation list;mutable weights:float list;mutable bias:float;alpha:float}
  3. type trainingRotationOutput = {weights:float list; bias:float}
  4. type predictionRotationOutput = {observation: observation;yActual:float}
  5.  
  6. let runTrainingRotation(rotationInput: rotationInput)=
  7.     for i=0 to rotationInput.observations.Length do
  8.         let observation = rotationInput.observations.[i]
  9.         let neuronInput = {cycleInput.xValues=observation.xValues;cycleInput.yExpected=observation.yExpected;cycleInput.weights=rotationInput.weights;
  10.                             cycleInput.bias=rotationInput.bias;cycleInput.alpha=rotationInput.alpha}
  11.         let cycleOutput = runCycle(neuronInput)
  12.         rotationInput.weights <- cycleOutput.weights
  13.         rotationInput.bias <- cycleOutput.bias
  14.     {weights=rotationInput.weights; bias=rotationInput.bias}

Again, note the liberal use of records to keep the inputs and outputs clear.  I also created a prediction rotation that is designed to be run only once that does not alter the weights and bias.

  1. let runPredictionRotation(rotationInput: rotationInput)=
  2.     let output = new System.Collections.Generic.List<predictionRotationOutput>()
  3.     for i=0 to rotationInput.observations.Length do
  4.         let observation = rotationInput.observations.[i]
  5.         let neuronInput = {cycleInput.xValues=observation.xValues;cycleInput.yExpected=observation.yExpected;cycleInput.weights=rotationInput.weights;
  6.                             cycleInput.bias=rotationInput.bias;cycleInput.alpha=rotationInput.alpha}
  7.         let cycleOutput = runCycle(neuronInput)
  8.         let predictionRotationOutput = {observation=observation;yActual=cycleOutput.yActual}
  9.         output.Add(predictionRotationOutput)   
  10.     output

With the rotations done, the last step was to create the Perceptron to train and then predict:

  1. type perceptronInput = {observations: observation list;weights:float list;bias:float}
  2. type perceptronOutput = {weights:float list; bias:float}
  3.  
  4. let initializeWeights(xValues, randomSeedValue) =
  5.     let lo = -0.01
  6.     let hi = 0.01
  7.     let xWeight = (hi-lo) * randomSeedValue + lo
  8.     xValues |> List.map(fun w -> xWeight)
  9.  
  10. let initializeBias(randomSeedValue) =
  11.     let lo = -0.01
  12.     let hi = 0.01
  13.     (hi-lo) * randomSeedValue + lo
  14.  
  15. let runTraining(perceptronInput: perceptronInput, maxEpoches:int) =
  16.     let random = System.Random()
  17.     let alpha = 0.001
  18.     let baseObservation = perceptronInput.observations.[0]
  19.     let mutable weights = initializeWeights(baseObservation.xValues,random.NextDouble())       
  20.     let mutable bias = initializeBias(random.NextDouble())
  21.     let rotationList = [0..maxEpoches]
  22.     for i=0 to maxEpoches do
  23.         let rotationInput = {observations=perceptronInput.observations;weights=weights;bias=bias;alpha=alpha}
  24.         let rotationOutput = runTrainingRotation(rotationInput)
  25.         weights <- rotationOutput.weights
  26.         bias <- rotationOutput.bias
  27.     {weights=weights;bias=bias}
  28.  
  29. let runPrediction(perceptronInput: perceptronInput, weights: float list, bias: float) =
  30.     let random = System.Random()
  31.     let alpha = 0.001
  32.     let rotationInput = {observations=perceptronInput.observations;weights=weights;bias=bias;alpha=alpha}
  33.     runPredictionRotation(rotationInput)

 

Before I go too much further, I have a big code smell.  I am iterating and using the mutable keyword.  I am not sure how to take the results of a function that is applied to the 1st element in a sequence and then input that into the second.  I need to do that with the weights and bias data structures –> each time it is used in a expression, it need to change and feed into the next expression.  I think the answer is the List.Reduce, so I am going to pick this up after looking at that in more detail.  I also need to implement the shuffle method so that that cycles are not called in the same order across rotations….

Filed under F#, Machine Learning

Neural Networks

July 15, 2014 1 Comment

I picked up James McCaffrey’s Neural Networks Using C# a couple of weeks ago and decided to see if I could rewrite the code in F#.  Unfortunately, the source code is not available (as far as I could tell), so I did some C# then F# coding to see if I could get functional equivalence.

My first stop was chapter one.  I made the decision to get the F# code working for the sample data that McCaffrey provided first and then refactor it to a more general program that would work with inputs and values of different datasets.  My final upgrade will be use Deedle instead of any other data structure.  But first things first, I want to get the examples working so I fired up a script file and opened my REPL.

McCaffrey defines a sample dataset like this

  1. string[] sourceData = new string[] { "Sex Age Locale Income Politics",
  2.     "==============================================",
  3.     "Male 25 Rural 63,000.00 Conservative",
  4.     "Female 36 Suburban 55,000.00 Liberal", "Male 40 Urban 74,000.00 Moderate",
  5.     "Female 23 Rural 28,000.00 Liberal" };

He then creates a parser for the comma-delimited string values into a double[][].  I just created the dataset as a List of tuples.

  1. let chapter1TestData = [("Male",25.,"Rural",63000.00,"Conservative");
  2.                 ("Female",36.,"Suburban",55000.00,"Liberal");
  3.                 ("Male",40.,"Urban",74000.00,"Moderate");
  4.                 ("Female",23.,"Rural",28000.00,"Liberal")]

 

I did try an implementation using a record type but for reasons below, I am using Tuples.  With the equivalent data loaded into  the REPL, I tackled the first supporting function: MinMax.  Here is the C# code that McCaffrey wrote:

  1. static void MinMaxNormal(double[][] data, int column)
  2. {
  3.     int j = column;
  4.     double min = data[0][j];
  5.     double max = data[0][j];
  6.     for (int i = 0; i < data.Length; ++i)
  7.     {
  8.         if (data[i][j] < min) min = data[i][j];
  9.         if (data[i][j] > max) max = data[i][j];
  10.     }
  11.     double range = max – min;
  12.     if (range == 0.0) // ugly
  13.     { for (int i = 0; i < data.Length; ++i)
  14.         data[i][j] = 0.5;
  15.         return; }
  16.     for (int i = 0; i < data.Length; ++i)
  17.         data[i][j] = (data[i][j] – min) / range;
  18. }

and here is the equivalent F# code.

  1. let minMax (fullSet, i) =
  2.     let min = fullSet |> Seq.min
  3.     let max = fullSet |> Seq.max
  4.     (i-min)/(max-min)

 

Note that McCaffrey does not have any unit tests but when I ran the dummy data through the F# implementation, the results matched his screen shots so that will work well enough.  If you ever need a reason to use F#, consider those 2 code samples.  Granted McCaffrey’s code is more abstract because it can be any column in double array, but my counterpoint is that the function is really doing too much and it is trivial in F# to pick a given column.  Is there any doubt what the F# code is doing?  Is there any certainty of what the C# code is doing?

In any event, moving along to the next functions, McCaffrey created two functions that do all of the encoding of the string values to appropriate numeric ones.  Depending on if the value is a X value (independent) or Y value (dependent), there is a different encoding scheme:

  1.  static string EffectsEncoding(int index, int N)
  2.  {
  3.      // If N = 3 and index = 0 -> 1,0.
  4.      // If N = 3 and index = 1 -> 0,1.
  5.      // If N = 3 and index = 2 -> -1,-1.
  6.      if (N == 2)
  7.      // Special case.
  8.      { if (index == 0) return "-1"; else if (index == 1) return "1"; }
  9.      int[] values = new int[N – 1];
  10.      if (index == N – 1)
  11.      // Last item is all -1s.
  12.      { for (int i = 0; i < values.Length; ++i) values[i] = -1; }
  13.      else
  14.      {
  15.          values[index] = 1;
  16.          // 0 values are already there.
  17.      } string s = values[0].ToString();
  18.      for (int i = 1; i < values.Length; ++i) s += "," + values[i]; return s;
  19.  }
  20.  
  21.  static string DummyEncoding(int index, int N)
  22.  {
  23.      int[] values = new int[N]; values[index] = 1;
  24.      string s = values[0].ToString();
  25.      for (int i = 1; i < values.Length; ++i) s += "," + values[i];
  26.      return
  27. }

In my F# project, I decided to domain-specific encoding.  I plan to refactor this to something more abstract. 

  1. //Transform Sex
  2. let testData' = chapter1TestData |> Seq.map(fun (s,a,l,i,p) -> match s with
  3.                                                                | "Male"-> -1.0,a,l,i,p
  4.                                                              | "Female" -> 1.0,a,l,i,p
  5.                                                              | _ -> failwith "Invalid sex")
  6. //Normalize Age
  7. let testData'' =
  8.     let fullSet =  testData' |> Seq.map(fun (s,a,l,i,p) -> a)
  9.     testData' |> Seq.map(fun (s,a,l,i,p) -> s,minMax(fullSet,a),l,i,p)
  10.  
  11. //Transform Locale
  12. let testData''' = testData'' |> Seq.map(fun (s,a,l,i,p) -> match l with
  13.                                                                 | "Rural" -> s,a,1.,0.,i,p
  14.                                                                 | "Suburban" -> s,a,0.,1.,i,p
  15.                                                                 | "Urban" -> s,a,-1.,-1.,i,p
  16.                                                                 | _ -> failwith "Invalid locale")
  17. //Transform and Normalize Income
  18. let testData'''' =
  19.     let fullSet =  testData''' |> Seq.map(fun (s,a,l0,l1,i,p) -> i)
  20.     testData''' |> Seq.map(fun (s,a,l0,l1,i,p) -> s,a,l0,l1,minMax(fullSet,i),p)
  21.  
  22. //Transform Politics
  23. let testData''''' = testData'''' |> Seq.map(fun (s,a,l0,l1,i,p) -> match p with
  24.                                                                 | "Conservative" -> s,a,l0,l1,i,1.,0.,0.
  25.                                                                 | "Liberal" -> s,a,l0,l1,i,0.,1.,0.
  26.                                                                 | "Moderate" -> s,a,l0,l1,i,0.,0.,1.
  27.                                                                 | _ -> failwith "Invalid politics")

When I execute the script:

image

Which is the same as McCaffrey’s.

image

Note that he used Gaussian normalization on column 2 and I did Min/Max based on his advice in the book.

 

 

Filed under Analytics, F#

Terminator Program: Part 2

July 8, 2014 1 Comment

Following up on my last post, I decided to send the entire photograph to Sky Biometry and have them parse the photograph and identify individual people.  This ability is built right into their API.  For example, if you pass them this picture, you get the following json back.

image

I added the red highlight to show that Sky Biometry can recognize multiple people (it is an array of uids) and that each face tag has a center.x and center:y.  Reading the API documentation, this point is center of the face tag point and their point is a percentage of the photo width.

image

So I need to translate the center point of the skeleton from the Kinect to eqiv center point of the sky biometry recognition output and I should be able to identify individual people within the Kinect’s field of vision.  Going back to the Kinect code, I ditched the DrawBoxAroundHead method and altered the UpdateDisplay method like so

  1. private void UpdateDisplay(byte[] colorData, Skeleton[] skeletons)
  2. {
  3.     if (_videoBitmap == null)
  4.     {
  5.         _videoBitmap = new WriteableBitmap(640, 480, 96, 96, PixelFormats.Bgr32, null);
  6.     }
  7.     _videoBitmap.WritePixels(new Int32Rect(0, 0, 640, 480), colorData, 640 * 4, 0);
  8.     kinectColorImage.Source = _videoBitmap;
  9.     var selectedSkeleton = skeletons.FirstOrDefault(s => s.TrackingState == SkeletonTrackingState.Tracked);
  10.     if (selectedSkeleton != null)
  11.     {
  12.         var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  13.         var adjustedHeadPosition =
  14.             _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  15.         var adjustedSkeletonPosition = _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(selectedSkeleton.Position, ColorImageFormat.RgbResolution640x480Fps30);
  16.  
  17.         skeletonCanvas.Children.Clear();
  18.         Rectangle headRectangle = new Rectangle();
  19.         headRectangle.Fill = new SolidColorBrush(Colors.Blue);
  20.         headRectangle.Width = 10;
  21.         headRectangle.Height = 10;
  22.         Canvas.SetLeft(headRectangle, adjustedHeadPosition.X);
  23.         Canvas.SetTop(headRectangle, adjustedHeadPosition.Y);
  24.         skeletonCanvas.Children.Add(headRectangle);
  25.  
  26.         Rectangle skeletonRectangle = new Rectangle();
  27.         skeletonRectangle.Fill = new SolidColorBrush(Colors.Red);
  28.         skeletonRectangle.Width = 10;
  29.         skeletonRectangle.Height = 10;
  30.         Canvas.SetLeft(skeletonRectangle, adjustedHeadPosition.X);
  31.         Canvas.SetTop(skeletonRectangle, adjustedHeadPosition.Y);
  32.         skeletonCanvas.Children.Add(skeletonRectangle);
  33.  
  34.         String skeletonInfo = headPosition.X.ToString() + " : " + headPosition.Y.ToString() + " — ";
  35.         skeletonInfo = skeletonInfo + adjustedHeadPosition.X.ToString() + " : " + adjustedHeadPosition.Y.ToString() + " — ";
  36.         skeletonInfo = skeletonInfo + adjustedSkeletonPosition.X.ToString() + " : " + adjustedSkeletonPosition.Y.ToString();
  37.  
  38.         skeletonInfoTextBox.Text = skeletonInfo;
  39.  
  40.     }
  41. }

Notice that there are two rectangles because I was not sure if the Head.Position or the Skeleton.Position would match SkyBiometry.  Turns out that I want the Head.Position for SkyBiometry (besides, the terminator would want head shots only)

image

So I ditched the Skeleton.Position.  I then needed a way to translate the Head.Posotion.X to SkyBiometry.X and Head.Posotion.Y to SkyBiometry.Y.  Fortunately, I know the size of each photograph (640 X 480) so calculating the percent is an exercise of altering UpdateDisplay:

  1. private void UpdateDisplay(byte[] colorData, Skeleton[] skeletons)
  2. {
  3.     Int32 photoWidth = 640;
  4.     Int32 photoHeight = 480;
  5.  
  6.     if (_videoBitmap == null)
  7.     {
  8.         _videoBitmap = new WriteableBitmap(photoWidth, photoHeight, 96, 96, PixelFormats.Bgr32, null);
  9.     }
  10.     _videoBitmap.WritePixels(new Int32Rect(0, 0, photoWidth, photoHeight), colorData, photoWidth * 4, 0);
  11.     kinectColorImage.Source = _videoBitmap;
  12.     var selectedSkeleton = skeletons.FirstOrDefault(s => s.TrackingState == SkeletonTrackingState.Tracked);
  13.     if (selectedSkeleton != null)
  14.     {
  15.         var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  16.         var adjustedHeadPosition =
  17.             _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  18.  
  19.         skeletonCanvas.Children.Clear();
  20.         Rectangle headRectangle = new Rectangle();
  21.         headRectangle.Fill = new SolidColorBrush(Colors.Blue);
  22.         headRectangle.Width = 10;
  23.         headRectangle.Height = 10;
  24.         Canvas.SetLeft(headRectangle, adjustedHeadPosition.X);
  25.         Canvas.SetTop(headRectangle, adjustedHeadPosition.Y);
  26.         skeletonCanvas.Children.Add(headRectangle);
  27.  
  28.         var skyBiometryX = ((float)adjustedHeadPosition.X / photoWidth)*100;
  29.         var skyBioMetryY = ((float)adjustedHeadPosition.Y / photoHeight)*100;
  30.  
  31.         String skeletonInfo = adjustedHeadPosition.X.ToString() + " : " + adjustedHeadPosition.Y.ToString() + " — ";
  32.         skeletonInfo = skeletonInfo + Math.Round(skyBiometryX,2).ToString() + " : " + Math.Round(skyBioMetryY,2).ToString();
  33.  
  34.         skeletonInfoTextBox.Text = skeletonInfo;
  35.  
  36.     }

And so now I have

image

The next step is to get the Kinect photo to Sky Biometry.  I decided to use Azure Blob Storage as my intermediately location.  I updated the architectural diagram like so:

image

At this point, it made sense to move the project over to F# so I could better concentrate on the work that needs to be done and also getting the important code out of the UI code behind.  I fired up a F# project in my solution added a couple different implementations of Storing Photos.  To keep things consistent, I created a data structure and an interface:

  1. namespace ChickenSoftware.Terminator.Core
  2.  
  3. open System
  4.  
  5. type public PhotoImage (uniqueId:Guid, imageBytes:byte[]) =
  6.     member this.UniqueId = uniqueId
  7.     member this.ImageBytes = imageBytes
  8.  
  9. type IPhotoImageProvider =
  10.     abstract member InsertPhotoImage : PhotoImage -> unit
  11.     abstract member DeletePhotoImage : Guid -> unit
  12.     abstract member GetPhotoImage : Guid -> PhotoImage

My 1st stop was to replicate what Miles did with the Save File Dialog box with a File System Provider.  It was very much like a C# implementation:

  1. namespace ChickenSoftware.Terminator.Core
  2.  
  3. open System
  4. open System.IO
  5. open System.Drawing
  6. open System.Drawing.Imaging
  7.  
  8. type LocalFileSystemPhotoImageProvider(folderPath: string) =
  9.  
  10.     member this.GetPhotoImageUri(uniqueIdentifier: Guid) =
  11.         let fileName = uniqueIdentifier.ToString() + ".jpg"
  12.         Path.Combine(folderPath, fileName)
  13.  
  14.     interface IPhotoImageProvider with
  15.         member this.InsertPhotoImage(photoImage: PhotoImage) =
  16.             let fullPath = this.GetPhotoImageUri(photoImage.UniqueId)
  17.             use memoryStream = new MemoryStream(photoImage.ImageBytes)
  18.             let image = Image.FromStream(memoryStream)
  19.             image.Save(fullPath)
  20.  
  21.         member this.DeletePhotoImage(uniqueIdentifier: Guid) =
  22.             let fullPath = this.GetPhotoImageUri(uniqueIdentifier)
  23.             File.Delete(fullPath)        
  24.  
  25.         member this.GetPhotoImage(uniqueIdentifier: Guid) =
  26.             let fullPath = this.GetPhotoImageUri(uniqueIdentifier)
  27.             use fileStream = new FileStream(fullPath,FileMode.Open)
  28.             let image = Image.FromStream(fileStream)
  29.             use memoryStream = new MemoryStream()
  30.             image.Save(memoryStream,ImageFormat.Jpeg)
  31.             new PhotoImage(uniqueIdentifier, memoryStream.ToArray())

To call the save method, I altered the SavePhoto method in the C# project to use a MemoryStream and not a FileStream:

  1. private void SavePhoto(byte[] colorData)
  2. {
  3.     var bitmapSource = BitmapSource.Create(640, 480, 96, 96, PixelFormats.Bgr32, null, colorData, 640 * 4);
  4.     JpegBitmapEncoder encoder = new JpegBitmapEncoder();
  5.     encoder.Frames.Add(BitmapFrame.Create(bitmapSource));
  6.     using (MemoryStream memoryStream = new MemoryStream())
  7.     {
  8.         encoder.Save(memoryStream);
  9.         PhotoImage photoImage = new PhotoImage(Guid.NewGuid(), memoryStream.ToArray());
  10.  
  11.         String folderUri = @"C:\Data";
  12.         IPhotoImageProvider provider = new LocalFileSystemPhotoImageProvider(folderUri);
  13.  
  14.         provider.InsertPhotoImage(photoImage);
  15.         memoryStream.Close();
  16.     }
  17.     _isTakingPicture = false;
  18. }

And sure enough, it saves the photo to disk:

image

One problem that took me 20 minutes to uncover is that if you get your file system path wrong, you get the unhelpful exception:

image

This has been well-bitched about on stack overflow so I won’t comment further. 

With the file system up and running, I turned my attention to Azure.  Like the File System provider, it is very close to a C# implementation

  1. namespace ChickenSoftware.Terminator.Core
  2.  
  3. open System
  4. open System.IO
  5. open Microsoft.WindowsAzure.Storage
  6. open Microsoft.WindowsAzure.Storage.Blob
  7.  
  8. type AzureStoragePhotoImageProvider(customerUniqueId: Guid, connectionString: string) =
  9.  
  10.     member this.GetBlobContainer(blobClient:Blob.CloudBlobClient) =
  11.         let container = blobClient.GetContainerReference(customerUniqueId.ToString())
  12.         if not (container.Exists()) then
  13.             container.CreateIfNotExists() |> ignore
  14.             let permissions = new BlobContainerPermissions()
  15.             permissions.PublicAccess <- BlobContainerPublicAccessType.Blob
  16.             container.SetPermissions(permissions)
  17.         container
  18.  
  19.     member this.GetBlockBlob(uniqueIdentifier: Guid) =
  20.         let storageAccount = CloudStorageAccount.Parse(connectionString)
  21.         let blobClient = storageAccount.CreateCloudBlobClient()
  22.         let container = this.GetBlobContainer(blobClient)
  23.         let photoUri = this.GetPhotoImageUri(uniqueIdentifier)
  24.         container.GetBlockBlobReference(photoUri)
  25.  
  26.     member this.GetPhotoImageUri(uniqueIdentifier: Guid) =
  27.         uniqueIdentifier.ToString() + ".jpg"
  28.  
  29.     interface IPhotoImageProvider with
  30.         member this.InsertPhotoImage(photoImage: PhotoImage) =
  31.             let blockBlob = this.GetBlockBlob(photoImage.UniqueId)
  32.             use memoryStream = new MemoryStream(photoImage.ImageBytes)
  33.             blockBlob.UploadFromStream(memoryStream)
  34.  
  35.         member this.DeletePhotoImage(uniqueIdentifier: Guid) =
  36.             let blockBlob = this.GetBlockBlob(uniqueIdentifier)
  37.             blockBlob.Delete()       
  38.  
  39.         member this.GetPhotoImage(uniqueIdentifier: Guid) =
  40.             let blockBlob = this.GetBlockBlob(uniqueIdentifier)
  41.             if blockBlob.Exists() then
  42.                 blockBlob.FetchAttributes()
  43.                 use memoryStream = new MemoryStream()
  44.                 blockBlob.DownloadToStream(memoryStream)
  45.                 let photoArray = memoryStream.ToArray()
  46.                 new PhotoImage(uniqueIdentifier,photoArray)
  47.             else
  48.                 failwith "photo not found"

And when I pop it into the WPF application,

  1. private void SavePhoto(byte[] colorData)
  2. {
  3.     var bitmapSource = BitmapSource.Create(640, 480, 96, 96, PixelFormats.Bgr32, null, colorData, 640 * 4);
  4.     JpegBitmapEncoder encoder = new JpegBitmapEncoder();
  5.     encoder.Frames.Add(BitmapFrame.Create(bitmapSource));
  6.     using (MemoryStream memoryStream = new MemoryStream())
  7.     {
  8.         encoder.Save(memoryStream);
  9.         PhotoImage photoImage = new PhotoImage(Guid.NewGuid(), memoryStream.ToArray());
  10.  
  11.         Guid customerUniqueId = new Guid("7282AF48-FB3D-489B-A572-2EFAE80D0A9E");
  12.         String connectionString =
  13.             "DefaultEndpointsProtocol=http;AccountName=XXX;AccountKey=XXX";
  14.         IPhotoImageProvider provider = new AzureStoragePhotoImageProvider(customerUniqueId, connectionString);
  15.  
  16.  
  17.         provider.InsertPhotoImage(photoImage);
  18.         memoryStream.Close();
  19.     }
  20.     _isTakingPicture = false;
  21. }

I can now write my images to Azure.

image

With that out of the way, I can now have SkyBiometry pick up my photo, analyze it, and push the results back.  I went ahead and added in the .fs module that I had already created for this blog post.  I then added FSharp.Data via NuGet and was ready to roll. In he Save photo event handler,after saving the photo to blob storage, it then calls Sky Biometry to compare against a base image that has already been trained:

  1. private void SavePhoto(byte[] colorData)
  2. {
  3.     var bitmapSource = BitmapSource.Create(640, 480, 96, 96, PixelFormats.Bgr32, null, colorData, 640 * 4);
  4.     JpegBitmapEncoder encoder = new JpegBitmapEncoder();
  5.     encoder.Frames.Add(BitmapFrame.Create(bitmapSource));
  6.     PhotoImage photoImage = UploadPhotoImage(encoder);
  7.  
  8.     String skyBiometryUri = "http://api.skybiometry.com&quot;;
  9.     String uid = "Kinect@ChickenFace";
  10.     String apiKey = "XXXX";
  11.     String apiSecret = "XXXX";
  12.  
  13.     var imageComparer = new SkyBiometryImageComparer(skyBiometryUri, uid, apiKey, apiSecret);
  14.     String basePhotoUri = "XXXX.jpg";
  15.     String targetPhotoUri = "XXXX/" + photoImage.UniqueId + ".jpg";
  16.  
  17.     currentImage.Source = new BitmapImage(new Uri(basePhotoUri));
  18.     compareImage.Source = new BitmapImage(new Uri(targetPhotoUri)); ;
  19.     
  20.     var matchValue = imageComparer.CalculateFacialRecognitionConfidence(basePhotoUri, targetPhotoUri);
  21.     FacialRecognitionTextBox.Text = "Match Value is: " + matchValue.ToString();
  22.     _isTakingPicture = false;
  23. }

And I am getting a result back from Sky Biometry.

image

Finally, I added in the SkyBiometry X and Y coordinates for the photo and compared to the calculated ones based on the Kinect Skeleton Tracking:

  1. currentImage.Source = new BitmapImage(new Uri(basePhotoUri));
  2. compareImage.Source = new BitmapImage(new Uri(targetPhotoUri)); ;
  3.  
  4. var matchValue = imageComparer.CalculateFacialRecognitionConfidence(basePhotoUri, targetPhotoUri);
  5.  
  6. var selectedSkeleton = skeletons.FirstOrDefault(s => s.TrackingState == SkeletonTrackingState.Tracked);
  7. if (selectedSkeleton != null)
  8. {
  9.     var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  10.     var adjustedHeadPosition =
  11.         _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  12.  
  13.     var skyBiometryX = ((float)adjustedHeadPosition.X / 640) * 100;
  14.     var skyBioMetryY = ((float)adjustedHeadPosition.Y / 480) * 100;
  15.  
  16.     StringBuilder stringBuilder = new StringBuilder();
  17.     stringBuilder.Append("Match Value is: ");
  18.     stringBuilder.Append(matchValue.Confidence.ToString());
  19.     stringBuilder.Append("Sky Biometry X: ");
  20.     stringBuilder.Append(matchValue.X.ToString());
  21.     stringBuilder.Append("Sky Biometry Y: ");
  22.     stringBuilder.Append(matchValue.Y.ToString());
  23.     stringBuilder.Append("Kinect X: ");
  24.     stringBuilder.Append(Math.Round(skyBiometryX, 2).ToString());
  25.     stringBuilder.Append("Kinect Y: ");
  26.     stringBuilder.Append(Math.Round(skyBioMetryY, 2).ToString());
  27.     FacialRecognitionTextBox.Text = stringBuilder.ToString();
  28. }
  29.  
  30. _isTakingPicture = false;

And the results are encouraging –> it looks like I can use the X and Y to identify different people on the screen:

Match Value is: 53
Sky Biometry X: 10
Sky Biometry Y: 13.33

Kinect X: 47.5
Kinect Y: 39.79

Up next will be pointing the laser and the target…

 

 

 

Filed under C#, F#, Kinect, WPF

Terminator Program: Part 1

July 8, 2014 Leave a comment

I am starting to work on a new Kinect application for TRINUG’s code camp.  I wanted to extend the facial recognition application I did using Sky Biometry and have the Kinect identify people in its field of view.  Then, I want to give the verbal command “Terminate XXX” where XXX is the name of a recognized person.  That would activate a couple of servos via a netduino and point a laser pointer at that person and perhaps make a blaster sound.  The <ahem> architectural diagram </ahem? looks like this

image

Not really worrying about how far I will get (the fun is in the process, no?), I picked up Rob Miles’s excellent book Start Here: Learn The Kinect API and plugged in my Kinect.

The first thing I did was see if I can get a running video from the Kinect –> which was very easy.  I created a new C#/WPF application and replaced the default markup with this::

  1. <Window x:Class="ChickenSoftware.Terminiator.UI.MainWindow"
  2.         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation&quot;
  3.         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot;
  4.         Title="MainWindow" Height="545" Width="643"
  5.         Loaded="Window_Loaded" Closing="Window_Closing">
  6.     <Grid>
  7.         <Image x:Name="kinectColorImage" Width="640" Height="480" />
  8.     </Grid>
  9. </Window>

And in the code-behind, I added the following code.  The only thing that is kinda tricky is that there are two threads: the Main UI thread and then the thread that processes the Kinect data.  Interestingly, it is easy to pass data from the Kinect Thread to the Main UI Thread –> just call the delegate and pass in the byte array.

  1. Boolean _isKinectDisplayActive = false;
  2. KinectSensor _sensor = null;
  3. WriteableBitmap _videoBitmap = null;
  4.  
  5. private void Window_Loaded(object sender, RoutedEventArgs e)
  6. {
  7.     SetUpKinect();
  8.     Thread videoThread = new Thread(new ThreadStart(DisplayKinectData));
  9.     _isKinectDisplayActive = true;
  10.     videoThread.Start();
  11. }
  13. private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
  14. {
  15.     _isKinectDisplayActive = false;
  16.  
  17. }
  18.  
  19. private void SetUpKinect()
  20. {
  21.     _sensor = KinectSensor.KinectSensors[0];
  22.     _sensor.ColorStream.Enable();
  23.     _sensor.Start();
  24. }
  25.  
  26. private void DisplayKinectData()
  27. {
  28.     while (_isKinectDisplayActive)
  29.     {
  30.         using (ColorImageFrame colorFrame = _sensor.ColorStream.OpenNextFrame(10))
  31.         {
  32.             if (colorFrame == null) continue;
  33.             var colorData = new byte[colorFrame.PixelDataLength];
  34.             colorFrame.CopyPixelDataTo(colorData);
  35.             Dispatcher.Invoke(new Action(() => UpdateDisplay(colorData)));
  36.         }
  37.     }
  38.     _sensor.Stop();
  39. }
  40.  
  41. private void UpdateDisplay(byte[] colorData)
  42. {
  43.     if (_videoBitmap == null)
  44.     {
  45.         _videoBitmap = new WriteableBitmap(640, 480, 96, 96, PixelFormats.Bgr32, null);
  46.     }
  47.     _videoBitmap.WritePixels(new Int32Rect(0, 0, 640, 480), colorData, 640 * 4, 0);
  48.     kinectColorImage.Source = _videoBitmap;
  49. }

And I have a live-feed video

image

With that out of the way, I went to add picture taking capability.  I altered the XAML like so:

  1. <Window x:Class="ChickenSoftware.Terminiator.UI.MainWindow"
  2.         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation&quot;
  3.         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot;
  4.         Title="MainWindow" Height="545" Width="643"
  5.         Loaded="Window_Loaded" Closing="Window_Closing">
  6.     <Grid>
  7.         <Image x:Name="kinectColorImage" Width="640" Height="480" />
  8.         <Button x:Name="takePhotoButton" Margin="0,466,435,10" Click="takePhotoButton_Click">Take Photo</Button>
  9.     </Grid>
  10. </Window>

And added this to the code behind:

  1. Boolean _isTakingPicture = false;
  2. BitmapSource _pictureBitmap = null;
  3.  
  4. private void takePhotoButton_Click(object sender, RoutedEventArgs e)
  5. {
  6.     _isTakingPicture = true;
  7.     SaveFileDialog dialog = new SaveFileDialog();
  8.     dialog.FileName = "Snapshot";
  9.     dialog.DefaultExt = ".jpg";
  10.     dialog.Filter = "Pictures (.jpg)|*.jpg";
  11.  
  12.     if (dialog.ShowDialog() == true)
  13.     {
  14.         String fileName = dialog.FileName;
  15.         using (FileStream fileStream = new FileStream(fileName, FileMode.Create))
  16.         {
  17.             JpegBitmapEncoder encoder = new JpegBitmapEncoder();
  18.             encoder.Frames.Add(BitmapFrame.Create(_pictureBitmap));
  19.             encoder.Save(fileStream);
  20.         }
  21.     }
  22. }

 

And altered the DisplayKinectDatra method to poll the _isTakingPicture flag

  1. private void DisplayKinectData()
  2. {
  3.     while (_isKinectDisplayActive)
  4.     {
  5.         using (ColorImageFrame colorFrame = _sensor.ColorStream.OpenNextFrame(10))
  6.         {
  7.             if (colorFrame == null) continue;
  8.             var colorData = new byte[colorFrame.PixelDataLength];
  9.             colorFrame.CopyPixelDataTo(colorData);
  10.             Dispatcher.Invoke(new Action(() => UpdateDisplay(colorData)));
  11.  
  12.             if (_isTakingPicture)
  13.             {
  14.                 Dispatcher.Invoke(new Action(() => SavePhoto(colorData)));
  15.             }
  16.         }
  17.     }
  18.     _sensor.Stop();
  19. }

And now I have screen capture ability.

image

With that out of the way, I needed a way of identifying the people in the Kinect’s field of vision and taking their picture individually.  I altered the XAML like so

  1. <Window x:Class="ChickenSoftware.Terminiator.UI.MainWindow"
  2.         xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation&quot;
  3.         xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml&quot;
  4.         Title="MainWindow" Height="545" Width="643"
  5.         Loaded="Window_Loaded" Closing="Window_Closing">
  6.     <Grid>
  7.         <Image x:Name="kinectColorImage" Width="640" Height="480" />
  8.         <Button x:Name="takePhotoButton" Margin="0,466,435,10" Click="takePhotoButton_Click">Take Photo</Button>
  9.         <Canvas x:Name="skeletonCanvas" Width="640" Height="480" />
  10.                 <TextBox x:Name="skeletonInfoTextBox" Margin="205,466,10,10" />
  11.     </Grid>
  12. </Window>

And altered the Setup method like so:

  1. private void SetUpKinect()
  2. {
  3.     _sensor = KinectSensor.KinectSensors[0];
  4.     _sensor.ColorStream.Enable();
  5.     _sensor.SkeletonStream.Enable();
  6.     _sensor.Start();
  7. }

And then altered the UpdateDisplay method to take in both the color byte array and the skeleton byte array and display the head and skeleton location.  Note that there is a built in function called MapSkeletonPointToColorPoint() which takes the skeleton coordinate position and translates it to the color coordinate position.  I know that is needed, but I have no idea who it works –> magic I guess.

  1. private void UpdateDisplay(byte[] colorData, Skeleton[] skeletons)
  2. {
  3.     if (_videoBitmap == null)
  4.     {
  5.         _videoBitmap = new WriteableBitmap(640, 480, 96, 96, PixelFormats.Bgr32, null);
  6.     }
  7.     _videoBitmap.WritePixels(new Int32Rect(0, 0, 640, 480), colorData, 640 * 4, 0);
  8.     kinectColorImage.Source = _videoBitmap;
  9.     var selectedSkeleton = skeletons.FirstOrDefault(s => s.TrackingState == SkeletonTrackingState.Tracked);
  10.     if (selectedSkeleton != null)
  11.     {
  12.         var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  13.         var adjustedHeadPosition =
  14.             _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  15.         var adjustedSkeletonPosition = _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(selectedSkeleton.Position, ColorImageFormat.RgbResolution640x480Fps30);
  16.  
  17.  
  18.         String skeletonInfo = headPosition.X.ToString() + " : " + headPosition.Y.ToString() + " — ";
  19.         skeletonInfo = skeletonInfo + adjustedHeadPosition.X.ToString() + " : " + adjustedHeadPosition.Y.ToString() + " — ";
  20.         skeletonInfo = skeletonInfo + adjustedSkeletonPosition.X.ToString() + " : " + adjustedSkeletonPosition.Y.ToString();
  21.  
  22.         skeletonInfoTextBox.Text = skeletonInfo;
  23.  
  24.     }
  25. }

And the invocation of the UpdateDisplay now looks like this:

  1. private void DisplayKinectData()
  2. {
  3.     while (_isKinectDisplayActive)
  4.     {
  5.         using (ColorImageFrame colorFrame = _sensor.ColorStream.OpenNextFrame(10))
  6.         {
  7.             if (colorFrame == null) continue;
  8.             using (SkeletonFrame skeletonFrame = _sensor.SkeletonStream.OpenNextFrame(10))
  9.             {
  10.                 if (skeletonFrame == null) continue;
  11.  
  12.                 var colorData = new byte[colorFrame.PixelDataLength];
  13.                 var skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
  14.  
  15.                 colorFrame.CopyPixelDataTo(colorData);
  16.                 skeletonFrame.CopySkeletonDataTo(skeletons);
  17.  
  18.  
  19.                 if (_isTakingPicture)
  20.                 {
  21.                     Dispatcher.Invoke(new Action(() => SavePhoto(colorData)));
  22.                 }
  23.                 Dispatcher.Invoke(new Action(() => UpdateDisplay(colorData, skeletons)));
  24.  
  25.             }
  26.         }
  27.     }
  28.     _sensor.Stop();
  29. }

And the results are what you expect:

image

With the ability to identify individuals, I then wants to take individual photos of each person and feed it to Sky Biometry.  To that end, I added a method to draw a rectangle around each person and then (somehow) take a snapshot of the contents within the triangle.  Drawing the rectangle was a straight-forward WPF exercise:

  1. private void DrawBoxAroundHead(Skeleton selectedSkeleton)
  2. {
  3.     skeletonCanvas.Children.Clear();
  4.     var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  5.     var shoulderCenterPosition = selectedSkeleton.Joints[JointType.ShoulderCenter].Position;
  6.  
  7.     var adjustedHeadPosition =
  8.         _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  9.     var adjustedShoulderCenterPosition =
  10.         _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(shoulderCenterPosition, ColorImageFormat.RgbResolution640x480Fps30);
  11.     var delta = adjustedHeadPosition.Y – adjustedShoulderCenterPosition.Y;
  12.     var centerX = adjustedHeadPosition.X;
  13.     var centerY = adjustedHeadPosition.Y;
  14.  
  15.     Line topLline = new Line();
  16.     topLline.Stroke = new SolidColorBrush(Colors.Red);
  17.     topLline.StrokeThickness = 5;
  18.     topLline.X1 = centerX + (delta * -1);
  19.     topLline.Y1 = centerY – (delta * -1);
  20.     topLline.X2 = centerX + delta;
  21.     topLline.Y2 = centerY – (delta * -1);
  22.     skeletonCanvas.Children.Add(topLline);
  23.     Line bottomLine = new Line();
  24.     bottomLine.Stroke = new SolidColorBrush(Colors.Red);
  25.     bottomLine.StrokeThickness = 5;
  26.     bottomLine.X1 = centerX + (delta * -1);
  27.     bottomLine.Y1 = centerY + (delta * -1);
  28.     bottomLine.X2 = centerX + delta;
  29.     bottomLine.Y2 = centerY + (delta * -1);
  30.     skeletonCanvas.Children.Add(bottomLine);
  31.     Line rightLine = new Line();
  32.     rightLine.Stroke = new SolidColorBrush(Colors.Red);
  33.     rightLine.StrokeThickness = 5;
  34.     rightLine.X1 = centerX + (delta * -1);
  35.     rightLine.Y1 = centerY – (delta * -1);
  36.     rightLine.X2 = centerX + (delta * -1);
  37.     rightLine.Y2 = centerY + (delta * -1);
  38.     skeletonCanvas.Children.Add(rightLine);
  39.     Line leftLine = new Line();
  40.     leftLine.Stroke = new SolidColorBrush(Colors.Red);
  41.     leftLine.StrokeThickness = 5;
  42.     leftLine.X1 = centerX + delta;
  43.     leftLine.Y1 = centerY – (delta * -1);
  44.     leftLine.X2 = centerX + delta;
  45.     leftLine.Y2 = centerY + (delta * -1);
  46.     skeletonCanvas.Children.Add(leftLine);
  47. }

And then adding that line in the Update Display

  1. if (selectedSkeleton != null)
  2. {
  3.     var headPosition = selectedSkeleton.Joints[JointType.Head].Position;
  4.     var adjustedHeadPosition =
  5.         _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(headPosition, ColorImageFormat.RgbResolution640x480Fps30);
  6.     var adjustedSkeletonPosition = _sensor.CoordinateMapper.MapSkeletonPointToColorPoint(selectedSkeleton.Position, ColorImageFormat.RgbResolution640x480Fps30);
  7.  
  8.     DrawBoxAroundHead(selectedSkeleton);
  9.  
  10.     String skeletonInfo = headPosition.X.ToString() + " : " + headPosition.Y.ToString() + " — ";
  11.     skeletonInfo = skeletonInfo + adjustedHeadPosition.X.ToString() + " : " + adjustedHeadPosition.Y.ToString() + " — ";
  12.     skeletonInfo = skeletonInfo + adjustedSkeletonPosition.X.ToString() + " : " + adjustedSkeletonPosition.Y.ToString();
  13.  
  14.     skeletonInfoTextBox.Text = skeletonInfo;
  15.  
  16. }

Gives me this:

image

Which is great, but now I am stuck.  I need a way of isolating the contents of that rectangle in the byte array that I am feeding to bitmap encoder and I don’t know how to trim the array.  Instead of trying to learn any more WPF and graphic programming, I decided to take a different tact and send the photograph in its entirety to Sky Biometry and let it figure out the people in the photograph.  How I did that is the subject of my next blog post…

 

 

 

 

Filed under C#, Kinect, WPF

TRINUG F# Analytics Prep: Part 2

July 1, 2014 1 Comment

I finished up the second part of the F#/Analytics lab scheduled for August.  It is a continuation of going through Astborg’s F# for Quantitative Finance that we started last month.  Here is my fist blog post on it.

In this lab, we are going to tackle the more advanced statistical calculations: the Black-Scholes formula, the Greeks, and Monte Carlo simulation. Using the same solution and projects, I started the script file to figure out the Black Scholes formula.  Astborg uses a couple of supporting functions which I knocked out first: Power and CumulativeDistribution.  I first created his function verbatim like this:

  1. let pow x n = exp(n*log(x))

and then refactored it to make it more readable like this

  1. let power baseNumber exponent = exp(exponent * log(baseNumber))

and then I realized it is the same thing as using pown which is already found in FSharp.Core. 

image

In any event, I then attacked the cumulativeDistribution method.  I downloaded the source from his website and then refactored it so that each step is clearly laid out.  Here is the refactored function:

  1. let cumulativeDistribution (x) =
  2.         let a1 =  0.31938153
  3.         let a2 = -0.356563782
  4.         let a3 =  1.781477937
  5.         let a4 = -1.821255978
  6.         let a5 =  1.330274429
  7.         let pi = 3.141592654
  8.         let l  = abs(x)
  9.         let k  = 1.0 / (1.0 + 0.2316419 * l)
  10.  
  11.         let a1' = a1*k
  12.         let a2' = a2*k*k
  13.         let a3' = a3*(power k 3.0)
  14.         let a4' = a4*(power k 4.0)
  15.         let a5' = a5*(power k 5.0)
  16.         let w1 = 1.0/sqrt(2.0*pi)
  17.         let w2 = exp(-l*l/2.0)
  18.         let w3 = a1'+a2'+a3'+a4'+a5'
  19.         let w  = 1.0-w1*w2*w3
  20.         if x < 0.0 then 1.0 – w else w

And here is some test values from the REPL:

image

Finally, the Black Scholes formula.  I did create a separate POCO for the input data like this:

  1. type putCallFlag = Put | Call
  2.  
  3. type blackScholesInputData =
  4.     {stockPrice:float;
  5.     strikePrice:float;
  6.     timeToExpiry:float;
  7.     interestRate:float;
  8.     volatility:float}

And I refactored his code to make it more readable like this:

  1. let blackScholes(inputData:blackScholesInputData, putCallFlag:putCallFlag)=
  2.    let sx = log(inputData.stockPrice / inputData.strikePrice)
  3.    let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  4.    let rvt = rv*inputData.timeToExpiry
  5.    let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  6.    let d1=(sx + rvt)/vt
  7.    let d2=d1-vt
  8.     
  9.    match putCallFlag with
  10.     | Put ->
  11.         let xrt = inputData.strikePrice*exp(-inputData.interestRate*inputData.timeToExpiry)
  12.         let cdD1 = xrt*cumulativeDistribution(-d2)
  13.         let cdD2 = inputData.stockPrice*cumulativeDistribution(-d1)
  14.         cdD1-cdD2
  15.     | Call ->
  16.         let xrt = inputData.strikePrice*exp(-inputData.interestRate*inputData.timeToExpiry)
  17.         let cdD1 = inputData.stockPrice*cumulativeDistribution(d1)
  18.         let cdD2 = xrt*cumulativeDistribution(d2)
  19.         cdD1-cdD2

And since I was in the script environment, I put in test data that matches the sample that Astborg used in the book:

  1. let inputData = {stockPrice=58.60;strikePrice=60.;timeToExpiry=0.5;interestRate=0.01;volatility=0.3}
  2. let runBSCall = blackScholes(inputData,Call)
  3. let runBSPut = blackScholes(inputData,Put)

And voila, the results match the book:

image

With the Black-Scholes out of the way, I then implemented the Greeks.  Note that I did add helper functions for clarity, and the results match the book:

  1. let blackScholesDelta (inputData:blackScholesInputData, putCallFlag:putCallFlag) =
  2.     let sx = log(inputData.stockPrice / inputData.strikePrice)
  3.     let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  4.     let rvt = rv*inputData.timeToExpiry
  5.     let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  6.     let d1=(sx + rvt)/vt
  7.     match putCallFlag with
  8.     | Put -> cumulativeDistribution(d1) – 1.0
  9.     | Call -> cumulativeDistribution(d1)
  10.  
  11. let deltaPut = blackScholesDelta(inputData, Put)
  12. let deltaCall = blackScholesDelta(inputData, Call)
  13.  
  14. let blackScholesGamma (inputData:blackScholesInputData) =
  15.     let sx = log(inputData.stockPrice / inputData.strikePrice)
  16.     let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  17.     let rvt = rv*inputData.timeToExpiry
  18.     let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  19.     let d1=(sx + rvt)/vt
  20.     normalDistribution.Density(d1)
  21.  
  22. let gamma = blackScholesGamma(inputData)
  23.  
  24. let blackScholesVega (inputData:blackScholesInputData) =
  25.     let sx = log(inputData.stockPrice / inputData.strikePrice)
  26.     let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  27.     let rvt = rv*inputData.timeToExpiry
  28.     let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  29.     let d1=(sx + rvt)/vt   
  30.     inputData.stockPrice*normalDistribution.Density(d1)*sqrt(inputData.timeToExpiry)
  31.  
  32. let vega = blackScholesVega(inputData)
  33.  
  34. let blackScholesTheta (inputData:blackScholesInputData, putCallFlag:putCallFlag) =
  35.     let sx = log(inputData.stockPrice / inputData.strikePrice)
  36.     let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  37.     let rvt = rv*inputData.timeToExpiry
  38.     let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  39.     let d1=(sx + rvt)/vt   
  40.     let d2=d1-vt
  41.     match putCallFlag with
  42.     | Put ->
  43.         let ndD1 = inputData.stockPrice*normalDistribution.Density(d1)*inputData.volatility
  44.         let ndD1' = ndD1/(2.0*sqrt(inputData.timeToExpiry))
  45.         let rx = inputData.interestRate*inputData.strikePrice
  46.         let rt = exp(-inputData.interestRate*inputData.timeToExpiry)
  47.         let cdD2 = rx*rt*cumulativeDistribution(-d2)
  48.         -(ndD1')+cdD2
  49.     | Call ->
  50.         let ndD1 = inputData.stockPrice*normalDistribution.Density(d1)*inputData.volatility
  51.         let ndD1' = ndD1/(2.0*sqrt(inputData.timeToExpiry))
  52.         let rx = inputData.interestRate*inputData.strikePrice
  53.         let rt = exp(-inputData.interestRate*inputData.timeToExpiry)
  54.         let cdD2 = cumulativeDistribution(d2)
  55.         -(ndD1')-rx*rt*cdD2
  56.  
  57. let thetaPut = blackScholesTheta(inputData, Put)
  58. let thetaCall = blackScholesTheta(inputData, Call)
  59.  
  60. let blackScholesRho (inputData:blackScholesInputData, putCallFlag:putCallFlag) =
  61.     let sx = log(inputData.stockPrice / inputData.strikePrice)
  62.     let rv = inputData.interestRate+inputData.volatility*inputData.volatility*0.5
  63.     let rvt = rv*inputData.timeToExpiry
  64.     let vt = (inputData.volatility*sqrt(inputData.timeToExpiry))
  65.     let d1=(sx + rvt)/vt   
  66.     let d2=d1-vt
  67.     match putCallFlag with
  68.     | Put ->
  69.         let xt = inputData.strikePrice*inputData.timeToExpiry
  70.         let rt = exp(-inputData.interestRate*inputData.timeToExpiry)  
  71.         -xt*rt*cumulativeDistribution(-d2)
  72.     | Call ->
  73.         let xt = inputData.strikePrice*inputData.timeToExpiry
  74.         let rt = exp(-inputData.interestRate*inputData.timeToExpiry)          
  75.         xt*rt*cumulativeDistribution(d2)
  76.  
  77. let rhoPut = blackScholesRho(inputData, Put)
  78. let rhoCall = blackScholesRho(inputData, Call)

 

image

Finally, I threw in the Monte Carlo, which also used a POCO:

  1. type monteCarloInputData =
  2.     {stockPrice:float;
  3.     strikePrice:float;
  4.     timeToExpiry:float;
  5.     interestRate:float;
  6.     volatility:float}
  7.  
  8. let priceAtMaturity (inputData:monteCarloInputData, randomValue:float) =
  9.     let s = inputData.stockPrice
  10.     let rv = (inputData.interestRate-inputData.volatility*inputData.volatility/2.0)
  11.     let rvt = rv*inputData.timeToExpiry
  12.     let vr = inputData.volatility*randomValue
  13.     let t = sqrt(inputData.timeToExpiry)
  14.     s*exp(rvt+vr*t)
  15.     
  16. let maturityPriceInputData = {stockPrice=58.60;strikePrice=60.0;timeToExpiry=0.5;interestRate=0.01;volatility=0.3}
  17. priceAtMaturity(maturityPriceInputData, 10.0)
  18.  
  19. let monteCarlo(inputData: monteCarloInputData, randomValues:seq<float>) =
  20.     randomValues
  21.         |> Seq.map(fun randomValue -> priceAtMaturity(inputData,randomValue) – inputData.strikePrice )
  22.         |> Seq.average
  23.  
  24.  
  25. let random = new System.Random()
  26. let rnd() = random.NextDouble()
  27. let data = [for i in 1 .. 1000 -> rnd() * 1.0]
  28.  
  29. let monteCarloInputData = {stockPrice=58.60;strikePrice=60.0;timeToExpiry=0.5;interestRate=0.01;volatility=0.3;}
  30. monteCarlo(monteCarloInputData,data)

image

One thing I really like about Astborg is that the Monte Carlo function does not new up the array of random numbers, rather they are passed in.  This makes the function much more testable and is the right way to right it (IMHO).  In fact, I think that seeing “new Random” or “DateTime.Now” hard-coded into functions is an anti-pattern that is all too common.

With the last of the functions done in the script file, I moved them into the .fs file and created covering unit tests based on the sample data that I did in the REPL.

  1. [TestMethod]
  2. public void PowerUsingValidData_ReturnsExpected()
  3. {
  4.     var calculations = new Calculations();
  5.     Double expected = 8;
  6.     Double actual = Math.Round(calculations.Power(2.0, 3.0), 0);
  7.     Assert.AreEqual(expected, actual);
  8. }
  9.  
  10. [TestMethod]
  11. public void CumulativeDistributionUsingValidData_ReturnsExpected()
  12. {
  13.     var calculations = new Calculations();
  14.     Double expected = .84134;
  15.     Double actual = Math.Round(calculations.CumulativeDistribution(1.0),5);
  16.     Assert.AreEqual(expected, actual);
  17. }
  18.  
  19. [TestMethod]
  20. public void BlackScholesCallUsingValidData_ReturnsExpected()
  21. {
  22.     var calculations = new Calculations();
  23.     Double expected = 4.4652;
  24.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  25.     Double actual = Math.Round(calculations.BlackScholes(inputData,PutCallFlag.Call), 5);
  26.     Assert.AreEqual(expected, actual);
  27. }
  28.  
  29. [TestMethod]
  30. public void BlackScholesPutUsingValidData_ReturnsExpected()
  31. {
  32.     var calculations = new Calculations();
  33.     Double expected = 5.56595;
  34.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  35.     Double actual = Math.Round(calculations.BlackScholes(inputData, PutCallFlag.Put), 5);
  36.     Assert.AreEqual(expected, actual);
  37. }
  38.  
  39. [TestMethod]
  40. public void DaysToYearsUsingValidData_ReturnsExpected()
  41. {
  42.     var calculations = new Calculations();
  43.     Double expected = .08214;
  44.     Double actual = Math.Round(calculations.DaysToYears(30), 5);
  45.     Assert.AreEqual(expected, actual);
  46. }
  47.  
  48. [TestMethod]
  49. public void BlackScholesDeltaCallUsingValidData_ReturnsExpected()
  50. {
  51.     var calculations = new Calculations();
  52.     Double expected = .50732;
  53.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  54.     Double actual = Math.Round(calculations.BlackScholesDelta(inputData, PutCallFlag.Call), 5);
  55.     Assert.AreEqual(expected, actual);
  56. }
  57.  
  58. [TestMethod]
  59. public void BlackScholesDeltaPutUsingValidData_ReturnsExpected()
  60. {
  61.     var calculations = new Calculations();
  62.     Double expected = -.49268;
  63.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  64.     Double actual = Math.Round(calculations.BlackScholesDelta(inputData, PutCallFlag.Put), 5);
  65.     Assert.AreEqual(expected, actual);
  66. }
  67.  
  68. [TestMethod]
  69. public void BlackScholesGammaUsingValidData_ReturnsExpected()
  70. {
  71.     var calculations = new Calculations();
  72.     Double expected = .39888;
  73.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  74.     Double actual = Math.Round(calculations.BlackScholesGamma(inputData), 5);
  75.     Assert.AreEqual(expected, actual);
  76. }
  77.  
  78. [TestMethod]
  79. public void BlackScholesVegaUsingValidData_ReturnsExpected()
  80. {
  81.     var calculations = new Calculations();
  82.     Double expected = 16.52798;
  83.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  84.     Double actual = Math.Round(calculations.BlackScholesVega(inputData), 5);
  85.     Assert.AreEqual(expected, actual);
  86. }
  87.  
  88. [TestMethod]
  89. public void BlackScholesThetaCallUsingValidData_ReturnsExpected()
  90. {
  91.     var calculations = new Calculations();
  92.     Double expected = -5.21103;
  93.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  94.     Double actual = Math.Round(calculations.BlackScholesTheta(inputData, PutCallFlag.Call), 5);
  95.     Assert.AreEqual(expected, actual);
  96. }
  97.  
  98. [TestMethod]
  99. public void BlackScholesThetaPutUsingValidData_ReturnsExpected()
  100. {
  101.     var calculations = new Calculations();
  102.     Double expected = -4.61402;
  103.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  104.     Double actual = Math.Round(calculations.BlackScholesTheta(inputData, PutCallFlag.Put), 5);
  105.     Assert.AreEqual(expected, actual);
  106. }
  107.  
  108. [TestMethod]
  109. public void BlackScholesRhoCallUsingValidData_ReturnsExpected()
  110. {
  111.     var calculations = new Calculations();
  112.     Double expected = 12.63174;
  113.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  114.     Double actual = Math.Round(calculations.BlackScholesRho(inputData, PutCallFlag.Call), 5);
  115.     Assert.AreEqual(expected, actual);
  116. }
  117.  
  118. [TestMethod]
  119. public void BlackScholesRhoPutUsingValidData_ReturnsExpected()
  120. {
  121.     var calculations = new Calculations();
  122.     Double expected = -17.21863;
  123.     var inputData = new BlackScholesInputData(58.6, 60.0, .5, .01, .3);
  124.     Double actual = Math.Round(calculations.BlackScholesRho(inputData, PutCallFlag.Put), 5);
  125.     Assert.AreEqual(expected, actual);
  126. }
  127.  
  128.  
  129. [TestMethod]
  130. public void PriceAtMaturityUsingValidData_ReturnsExpected()
  131. {
  132.     var calculations = new Calculations();
  133.     Double expected = 480.36923;
  134.     var inputData = new MonteCarloInputData(58.6, 60.0, .5, .01, .3);
  135.     Double actual = Math.Round(calculations.PriceAtMaturity(inputData, 10.0), 5);
  136.     Assert.AreEqual(expected, actual);
  137. }
  138.  
  139. [TestMethod]
  140. public void MonteCarloUsingValidData_ReturnsExpected()
  141. {
  142.     var calculations = new Calculations();
  143.     var inputData = new MonteCarloInputData(58.6, 60.0, .5, .01, .3);
  144.     var random = new System.Random();
  145.     List<Double> randomData = new List<double>();
  146.     for (int i = 0; i < 1000; i++)
  147.     {
  148.         randomData.Add(random.NextDouble());
  149.     }
  150.  
  151.     Double actual = Math.Round(calculations.MonteCarlo(inputData, randomData), 5);
  152.     var greaterThanFour = actual > 4.0;
  153.     var lessThanFive = actual < 5.0;
  154.  
  155.     Assert.AreEqual(true, greaterThanFour);
  156.     Assert.AreEqual(true, lessThanFive);
  157. }

 

With all of the tests running green, I then turned my attention to the UI.  I created more real state on the MainWindow  and added some additional data structures to the results of the analytics that lend themselves to charting and graphing.  For example:

  1. public class GreekData
  2. {
  3.     public Double StrikePrice { get; set; }
  4.     public Double DeltaCall { get; set; }
  5.     public Double DeltaPut { get; set; }
  6.     public Double Gamma { get; set; }
  7.     public Double Vega { get; set; }
  8.     public Double ThetaCall { get; set; }
  9.     public Double ThetaPut { get; set; }
  10.     public Double RhoCall { get; set; }
  11.     public Double RhoPut { get; set; }
  12.  
  13. }

And in the code behind of the MainWindow, I added some calcs based on the prior code that was already in it:

  1. var theGreeks = new List<GreekData>();
  2. for (int i = 0; i < 5; i++)
  3. {
  4.     var greekData = new GreekData();
  5.     greekData.StrikePrice = closestDollar – i;
  6.     theGreeks.Add(greekData);
  7.     greekData = new GreekData();
  8.     greekData.StrikePrice = closestDollar + i;
  9.     theGreeks.Add(greekData);
  10. }
  11. theGreeks.Sort((greek1,greek2)=>greek1.StrikePrice.CompareTo(greek2.StrikePrice));
  12.  
  13. foreach (var greekData in theGreeks)
  14. {
  15.     var inputData =
  16.         new BlackScholesInputData(adjustedClose, greekData.StrikePrice, .5, .01, .3);
  17.     greekData.DeltaCall = calculations.BlackScholesDelta(inputData, PutCallFlag.Call);
  18.     greekData.DeltaPut = calculations.BlackScholesDelta(inputData, PutCallFlag.Put);
  19.     greekData.Gamma = calculations.BlackScholesGamma(inputData);
  20.     greekData.RhoCall = calculations.BlackScholesRho(inputData, PutCallFlag.Call);
  21.     greekData.RhoPut = calculations.BlackScholesRho(inputData, PutCallFlag.Put);
  22.     greekData.ThetaCall = calculations.BlackScholesTheta(inputData, PutCallFlag.Call);
  23.     greekData.ThetaPut = calculations.BlackScholesTheta(inputData, PutCallFlag.Put);
  24.     greekData.Vega = calculations.BlackScholesVega(inputData);
  25.  
  26. }
  27.  
  28. this.TheGreeksDataGrid.ItemsSource = theGreeks;
  29.  
  30.  
  31. var blackScholes = new List<BlackScholesData>();
  32. for (int i = 0; i < 5; i++)
  33. {
  34.     var blackScholesData = new BlackScholesData();
  35.     blackScholesData.StrikePrice = closestDollar – i;
  36.     blackScholes.Add(blackScholesData);
  37.     blackScholesData = new BlackScholesData();
  38.     blackScholesData.StrikePrice = closestDollar + i;
  39.     blackScholes.Add(blackScholesData);
  40. }
  41. blackScholes.Sort((bsmc1, bsmc2) => bsmc1.StrikePrice.CompareTo(bsmc2.StrikePrice));
  42.  
  43. var random = new System.Random();
  44. List<Double> randomData = new List<double>();
  45. for (int i = 0; i < 1000; i++)
  46. {
  47.     randomData.Add(random.NextDouble());
  48. }
  49.  
  50. foreach (var blackScholesMonteCarlo in blackScholes)
  51. {
  52.     var blackScholesInputData =
  53.         new BlackScholesInputData(adjustedClose, blackScholesMonteCarlo.StrikePrice, .5, .01, .3);
  54.     var monteCarloInputData =
  55.         new MonteCarloInputData(adjustedClose, blackScholesMonteCarlo.StrikePrice, .5, .01, .3);
  56.  
  57.     blackScholesMonteCarlo.Call = calculations.BlackScholes(blackScholesInputData, PutCallFlag.Call);
  58.     blackScholesMonteCarlo.Put = calculations.BlackScholes(blackScholesInputData, PutCallFlag.Put);
  59.     blackScholesMonteCarlo.MonteCarlo = calculations.MonteCarlo(monteCarloInputData, randomData);
  60. }
  61.  
  62. this.BlackScholesDataGrid.ItemsSource = blackScholes;

And Whammo, the UI.

 

image

Fortunately Conrad D’Cruz is a member of TRINUG and an options trader and is going to explain what the heck we are looking at when the SIG gets together again.

 

Filed under Analytics, F#