Friday, September 16, 2016

Delphi And Kinect...

Recently I purchased a Kinect, originally from an X-Box 360 bundle but adapted for use on a computer. I'd been looking for a low cost animation solution for my PC for sometime in hopes of replacing the laborious process of hand animating 3D characters as in this animation video I produced a year and a half ago as a promo for one of my books, The Butterfly Dragon...


FYI, the guy in the end of the video who gets pummeled by her combo move is not Wing Chun Tai Chi JKD Master Wong by the way. It's actually a mercenary character from the upcoming sequel to The Butterfly Dragon who was hired to deal with her and a the notorious Dragon Butterfly.

The tools used to produce this sequence were: Daz 3D, Lightwave 3D, Adobe Photoshop, Hitfilm 3 and Magix Music Maker 2015. The music in this animation by the way was all composed from scratch (no samples or loops) on an M-Audio KeyRig 49.

All of the animation sequence at the end of that video depicting Ai Yuanlin Ying (Heylyn Yates) as the Butterfly Dragon in action was hand animated using forward kinematics with a little help from interpolation. Forward kinematics is a slow and logical approach to animation. For instance animating a walk cycle would require first moving the left upper leg forward a bit, the left lower leg backwards a bit, the left foot down a bit while moving the right upper leg back a bit and the lower leg forward a bit... etc. All of this is done in terms of arbitrary rotations of the pivotal joints on the body of the animated character. When combining this approach to animating the scene with the camera motions as well, there were some depth ordering challenges that required the scene and each of the characters to be rendered separately from one another, with the Butterfly Dragon's wings being rendered once as a black mask and then in full color so that the whole scene could be composited correctly with the combination of the camera motion. Overall it took seven passes of rendering the characters with the scene being rendered by itself (with the shadows) and then composited in post production.

Forward kinematics adds the ability to animate by moving the end point in a series of connected joints to a specific goal. So walking with inverse kinematics might involve starting out by moving the feet of the character (the rest of the leg joints would follow according to a series of constraints) and then adjusting the rest of the legs as desired, and then proceeding a few frames ahead in time and making another set of adjustments to progress the animation cycle. Everything in between that occurs as frames in the final animation output would be interpolated by the animation software itself.

Still a laborious process and if you're working with more than one character and coordinating their actions in response to one another, then animating with either forward or inverse kinematics can become a nightmare. Especially if you physically and intuitively know what you want. The problem then is interface between you and the character or scene that you're animating and essentially that's what makes animation such a talent and skill. If you don't have that talent or skill or the time, budget and/or patience to acquire it through instruction, practice and perseverance, then there's always motion capture.

There are several setups for operating a motion capture environment ranging from marker based video capture systems to sensor based networks and finally to markerless sensorless capture systems like the Microsoft Kinect. With a Kinect you can essentially track the rotations of the joints of the human body (up to 18 of them) and in close up mode it is possible to track facial motion making it the perfect tool for a low cost motion capture rig.

The Kinect's ability to capture skeletal and facial motion data is based upon a camera, a series of sensors (infrared, depth and a seldom used accelerometer), a microphone array for directional detection of a voice or audio source and some tricky internal hardware processing all delivered through a USB 2.0 interface in the X-Box 360 version. This Kinect is can be used in development projects through it's Windows based SDK which has matured at version 1.8 and is available for direct download from Microsoft. Make sure you download the SDK and the Kinect Developer tools as well.

The more recent X-Box One Kinect ups the anti a considerable amount by increasing the resolution of skeletal tracking to 26 joints (including accurate hip and hand tracking) and the resolution of the camera itself to 30 frames per second HD resolution.

One of the added advantages of each of these inexpensive capture solutions is the fact that they can also be used as a low resolution 3D scanner. That is, you could scan your face and then load the output 3D data recorded from the sensors into a modeling program like Lightwave 3D, Maya, Houdini, Blender or any of the other modeling programs available that can import standard Alias Wavefront obj files.

Now right out of the box if you buy used on Amazon or Ebay, you're not going to be in for a $60 motion capture solution. You're going to have to do a bit of research (most of which I've taken care of for you here) and you may even have to buy some additional software depending upon what you're trying to do. Not to mention the whole issue of motion acting. My animation requirements depend on a fair level of martial arts expertise which as a martial artist myself, I am not quite qualified to provide without breaking my neck or my walls. Preferably neither. Paying for someone else's skills can be costly if you're on a budget as well, so keep in mind that if you are trying to do any physical acting for motion capture to remember safety first.

The first places that you might want to look are at Dave Brekel's work with the Kinect via his website. This industrious individual has developed a number of applications, including a couple of free applications geared towards motion capture and 3D scanning using the Kinect. I didn't have much luck getting the free applications going with my system (thanks to the difficult process of setting up OpenNI drivers and the lack of support related to them though that appears to be improving). With his pay applications there is however no such barrier as Pro Point Cloud, Pro Body and Pro Face all interface with the Kinect through it's native Windows driver and SDK which works seamlessly provided your Kinect is installed correctly.

Using these tools and some others such as Adobe's Creative Suite, creativity, persistence and talent (inspiration?) you might come up with a great look and effect such as with this animation which utilized Kinect as one of its primary motion capture sources:



That still doesn't get to the title of my post, which is kind of misleading isn't it? What about Delphi and Kinect? Is there support for Kinect in the Delphi version of the latest Windows SDK? It turns out there is but with some catches. Jim McKeeth has done some work porting the Kinect SDK to Delphi but that conversion has stopped at version 1.5 of the Kinect SDK. There is much information about using Delphi and Kinect here at this originally great Spanish site which I share with you through a translation link so you may read it in your language of choice. As well there's much info here from this Korean site and once again translated to a language of your choice.

However, the Google code repository for the source branch of this version of the Delphi translation done by Jim McKeeth has vanished and that's because it has been relocated to GitHub under Embarcadero's repository there. Here's a direct link to the Kinect branch with the Delphi Headers and an example as well.

Following the example code from some of the Windows examples it is possible to interface the Kinect through Delphi, however there is a bit of reliance on Windows Event Objects which require a bit of work with threads and jumping through a few hoops to understand the workflow in working with the SDK to achieve goals such as timing data capture and streaming data formats from one of it's sensors.

I've been working with both Visual Studio and Delphi in an attempt to build a demo and utility application for the purposes of simple motion capture (without stepping on the toes of those who've already created some great applications that utilize Kinect). Perhaps in the near or somewhat near future I'll release these as I progress.

One thing that I will release is a recompiled set of binaries of the Kinect D2D Explorer application with the camera output reversed to the correct direction. In the Windows SDK example, the camera image is actually mirrored versus reality (meaning things on the left side of the room appear on the right etc). So in these versions I've rewritten it to correct that problem so that the camera output is no longer mirrored.

This zip file includes both the x86 and the x64 compiled versions on Google Drive.

The MD5 for the zip file is: 814583ac1955a9b105b42466170c5ee1

If you want the source, it's available at the ShhhhDigital Visual Studio Git repository.

Enjoy while I try to figure this out.

Brian Joseph Johns
http://thebutterflydragon.blogspot.ca
http://talesofthesanctum.blogspot.ca
http://shhhhdigital.blogspot.ca
http://delphitidbits.blogspot.ca

Friday, June 12, 2015

Singleton As A Simpleton...

One of the things in object orientation that cannot be gotten around easily is the often repetitive nature of the code you need to write to define objects, their properties and aggregates. Patterns came along relatively recently and addressed this issue by conceptualizing and packaging solutions to many common programming tasks in object orientation.

One such pattern is the Singleton pattern, a one way entry point to instantiate a variable and its memory, ensuring that it is only instanced once in the namespace where the pattern is defined. This ensures that cleanup is simple as well when the namespace containing the instance is freed.

In Delphi, the most common instantiation functions occur when instantiating an object property as in this example:

unit SingletonMyClass;

interface

uses Classes;

type
  TMyClass = class
  private
    FMyList: TList;
    function GetMyList: TList;
  public
    destructor Destroy; override;
    property MyList: TList read GetMyList;
  end;

implementation

function TMyClass.GetMyList: TList;
begin
  { Singleton getter }
  if not Assigned(FMyList) then
    FMyList := TList.Create;
  Result := FMyList;
end;

destructor TMyClass.Destroy;
begin
  FreeAndNil(FMyList);
  inherited;
end;

end.

In the above example, when we instance TMyClass and access the property MyList, the list will only be instanced once. When the TMyClass instance is freed, so will the one instance of the MyList field in TMyClass.

This is a fairly common arrangement in coding and unless you have code generators or an IDE that supports the creation of this common code, it will make up the bulk of your object oriented house keeping.

Of coding and design, efficiency of code can be achieved without compromising the art thereof and efficiency is something that we want if it makes the code more compact, and executes more quickly while getting the required task accomplished predictably.

One of the advantages of object orientation is that if a class exists that does part of the task we need to do or provides a base to build from to do it, we can simply create a descendant class from that class hence not reinventing the wheel. The VCL, Dot Net, and many other object oriented libraries exist for that purpose so lets take a look at how we can use this to reduce the code needed for such housekeeping tasks as handling the Singleton pattern when instancing an object property in a class.

Just for fun, lets call our simplified version the Simpleton:

unit MyLibrary;

interface

uses Classes;

type
  { Simpleton Object creation function }
  function SimpletonObject(AClass: TClass; AObject: TObject): TObject;
  
  { Simpleton Component creation function }
  function SimpletonComponent(AClass: TComponentClass; AComponent: TComponent; AOwner: 

implementation

function SimpletonObject(AClass: TClass; AObject: TObject): TObject;
begin
  if not Assigned(AObject) then
    AObject := AClass.Create;
  Result := AObject;
end;

function SimpletonComponent(AClass: TComponentClass; AComponent: TComponent; AOwner: TComponent): TComponent;
begin
  if not Assigned(AComponent) then
    AComponent := AComponent.Create(AOwner);
  Result := AComponent;

end;

end.


Now before you say: Are you out of your Vulcan mind! lets take a look at the ways this is used in code from the first example:


unit SingletonMyClass;

interface

uses Classes;

type
  TMyClass = class
  private
    FMyList: TList;
    function GetMyList: TList;
  public
    destructor Destroy; override;
    property MyList: TList read GetMyList;
  end;

implementation

function TMyClass.GetMyList: TList;
begin
  FMyList := TList(SimpletonObject(TList,FMyList));
  Result := FList;
end;

destructor TMyClass.Destroy;
begin
  FreeAndNil(FMyList);
  inherited;
end;

end.


Now once again, you might say: Big deal! We dropped one line of code and a comparison. No we didn't. We dropped one line of code everywhere you use a similar Singleton implementation and in compiled code that means several lines of machine code, not to mention the code to load the registers for the function call everywhere you use similar code. This means the code is more compact because the compiler won't just automatically eliminate all code that is redundant.

The only extra thing that we have to take into account is the extra typecast ie:

TList(SimpletonObject(TList,FMyList));

Experience teaches us the ways that we can shorten the distance to our goal.

Sharing those experiences with others can potentially shorten the distance of many to their similar goals.

Another aspect of doing things this way is that for component classes, we don't even need to apply it to a property getter because TComponent derived classes have their own ownership and instance management mechanism, so in a class that doesn't need to share access through a property getter publicly, you can use the Simpleton internally in a method to instance a Field that you might need access to privately. Hence no getter code (though object oriented purists might urge you to consider to limit such use due to the difficulty of finding issues with such code that instances outside of a property getter).

So in example once again:


unit SingletonMyComponent;

interface

uses Classes;

type
  TMyClass = class(TComponent)
  private
    FMyComponent: TComponent;
    procedure DoSomethingPrivate;
  public
    procedure DoSomething;
  end;

implementation

procedure TMyClass.DoSomethingPrivate;
begin
  FMyComponent := TComponent(SimpletonComponent(TComponent,FMyComponent,Self));

  // now do whatever it is we need to do with the FMyComponent instance...
end;

procedure TMyClass.DoSomething;
begin
  DoSomethingPrivate;
end;

end.

In the above example, we don't need property access if the Field is used internally as long as we remember to instance it with the SimpletonComponent function. If it is instanced already, it checks. Then when TMyClass if freed, the memory for FMyComponent is also automatically freed. Reduced code and reduce housekeeping.

Enjoy!

Brian Joseph Johns

Monday, November 24, 2014

Why Keep Your End Users Waiting? Try Multi-Threading With ShellExec API Calls...

Have you ever been using an application and decided to use the context sensitive help by pressing F1, only to be greeted by an indeterminately long pause while the application you were using waits for the help process to start?

Maybe an email link in an application that opens up your email client or web browser?

In this wonderful day and age of multicore high clock rate processors why does an application sit unrefreshed and essentially unresponsive while it waits for another task to start such as that of the help window, the email link or web browser link?

The truth is that it doesn't have to wait at all.

Let's take a look at the Win32 API call ShellExec for a second. A call to ShellExec runs in the application's main thread. Whenever you make a call to ShellExec that opens another process, it holds up the main VCL or application process. This is the same in many other commercial applications as well. Don't get me wrong, having the ability to run an application through an API call via a command line, so that the operating system determines how to handle the call is a great thing.

Now that we know the culprit holding up our little bit of time, what can we do?

One of the wonderful things about the VCL is encapsulation and fortunately it encapsulates threads quite well for this kind of situation. What we want to do is anytime we make a call to ShellExec, we instantiate it in a separate thread of its own, so that our application and more importantly our interface isn't held up while the user waits for the requested task to start. This is a computer after all and if it can do multiple things at once, shouldn't we be able to as well being one of the many wonderful marvels of life on this planet?

So using the Threads unit, we can simply implement a solution for any call to ShellExec call via the use of the following utility unit (tested with Delphi XE4 but should work with current and past versions as well):

{*******************************************************}
{                                                       }
{       ThreadedShellExec                               }
{                                                       }
{       Copyright (C) 2014 Brian Joseph Johns           }
{       Use and share freely                            }
{                                                       }
{*******************************************************}
unit ThreadedShellExec;

interface

uses
  System.Classes, Winapi.Windows;

type
  TShellExecThread = class(TThread)
  private
    { Private declarations }
    FCommandline: string;
  protected
    procedure Execute; override;
    property Commandline: string read FCommandline write FCommandline;
  end;

  procedure ThreadedShellExecute(ACommandline: string);

implementation

uses WinAPI.ShellAPI;

procedure ThreadedShellExecute(ACommandline: string);
begin
  with TShellExecThread.Create(True) do
  begin
    FreeOnTerminate := True;
    Commandline := ACommandline;
    {$if CompilerVersion >= 21}
    Start;
    {$else}
    Resume;
    {$endif}
  end;
end;

{ TShellExecThread }

procedure TShellExecThread.Execute;
begin
  { Place thread code here }
   ShellExecute(0,'open',PWideChar(FCommandline),nil,nil,SW_SHOWNORMAL);
end;

end.

Now, the next part is simple. We just replace our call to ShellExec with the above defined procedure ThreadedShellExecute.


Let's suppose that on a TLabel component on our application, we have defined an OnClick event that we want to open the end user's default web browser to a particular web site. Here's the OnClick handler that we might define:

implementation

uses ThreadedShellExec;

TForm1.Label1Click(Sender: TObject);
begin
  ThreadedShellExecute('http://delphitidbits.blogspot.ca');
end;

...

or alternately to open an email link and fill in the subject line for the end user:

implementation

uses ThreadedShellExec;

TForm1.Label1Click(Sender: TObject);
begin
  ThreadedShellExecute('mailto:fav.inbox@gmail.com?   subject=Requests%20And%20Feedback');
end;

As you can see, it simplifies the calling process and if you need more functionality from the ShellExec or to expand its functionality, it would require very little modification.

The best part is that when the end user clicks that label, they won't be kept waiting while their email client or browser loads.

Happy Coding.

Brian Joseph Johns
http://delphitidbits.blogspot.ca






Monday, November 17, 2014

Persistent Properties Without Hardcoding Part 1

Concepts: Properties, Array Properties, IniFiles, Run Time Type Information

Introduction

How many times have you sat down in the Delphi ide and code editor or in another specialized visual class editor and thought: Gee, if I encapsulated the process required to create properties, I could work a lot faster.

The truth is that you can encapsulate the process of creating properties for your form, component or any class that you'd like to be persistent. It only has a few requirements for your project or application.


  • Your application has file or registry access (preferably file access to its own directory).
  • Your application is not required to access these properties at high speed (as in a video game or graphics animation where accessing properties or fields might occur hundreds or even thousands of times per second).
  • The types you create are derived from simple types (ie not class or record types unless they are serializable into a simple type like a string).
We've given a hint that storage for the actual values for our properties will be kept in a file or the registry. That is precisely the case and in the case of using a file, specifically an Ini file, it allows us to make configuration changes before the GUI for your program is complete. So you can make all the important design tests without an interface at all because you can edit the state of properties from a text editor. The only GUI element you'll need is a means to signal a state change to the application itself if you've edited a property in such a manner. A button would do nicely for such purposes.

We will put all of the functionality for a property, just about any simple type based property in this property handler. In doing there will also be some unique advantages to this approach.

  • We will be able to defer the type specification until the property is needed, and not before.
  • No class specific code for property getters and setters. We can implement a single getter and setter to handle all properties that pass through the handler.
Array Properties

Array properties are a powerful language feature offered by Object Pascal and Delphi. They allow you to specify and index which is passed to a property getter or setter to handle the transaction.

ie

interface

TMyClass = class
private
  function GetIndexedProperty(Index: Integer): string;
  procedure SetIndexedProperty(Index: Integer; Const Value: string);
public
  property IndexedProperty[Index: Integer]: string read GetIndexedProperty write SetIndexedProperty;
end;

...

implementation

function TMyClass.GetIndexedProperty(Index: Integer): string;
begin
  // We don't have to get the value from an array.
  // We could retrieve it from a text file or from another object.
  Result := IntToStr(Index * 12);
end;

procedure TMyClass.GetIndexedProperty(Index: Integer; Const Value: string);
begin
  // We can do anything with Value that we want from here
  // There is no rule that says we have to store the value
  // or do anything with it
  MyArray[Index] := Value;
end;

In the above example, it is easy to see that we are using an Array Property much like we might in any application where the need arises.  When using Array Properties there is no rule that says that we have to use an Integer for the Index Specifier or even use the Index Specifier as the index into an array. It could be used for anything if we think a bit outside of the box. 

Take the following code as an example:


interface

TMyClass = class
private
  function GetIntegerProperty(Index: string): Integer;
  procedure SetIntegerProperty(Index: string; Const Value: Integer);
public
  property IntegerProperty[Index: Integer]: string read GetIntegerProperty write SetIntegerProperty;
end;

...

implementation

uses SysUtils, IniFiles;

function TMyClass.GetIntegerProperty(Index: string): Integer;
begin
  with TIniFile.Create(ChangeFileExt(Application.ExeName,'.ini')) do
  try
    Result := ReadInteger('Main',Index,0);
  finally
    Free;
  end;
end;

procedure TMyClass.GetIntegerProperty(Index: string; Const Value: Integer);
begin
  with TIniFile.Create(ChangeFileExt(Application.ExeName,'.ini')) do
  try
    Result := WriteInteger('Main',Index,Value);
  finally
    Free;
  end;
end;



Now in the example above, we've used the Index Specifier from the Array Property IntegerProperty to reference the ident parameter of a call to ReadInteger and WriteInteger, both are methods from the TIniFile class. In this case, we aren't using Index as a number at all but we are using it as an identifier. That means that our Index Specifier can act as a variable identifier. We could even define new properties at runtime using this mechanism as long as they were unique. However the program itself would only use such properties that it knew about at runtime, not the ones that you make up at runtime.

So take the following code using the above defined class:

...

const 
  PROPERTY_AGE  = 'Age';
  PROPERTY_YEAR = 'Year';

uses SysUtils, Dialogs;

procedure SomeProc;
begin
  with TMyClass.Create do
  try
    IntegerProperty[PROPERTY_AGE] := 47;
    IntegerProperty[PROPERTY_YEAR] := 2014;
    ShowMessage(Format('The property %s is %d',[PROPERTY_AGE,IntegerProperty[PROPERTY_AGE]]));
    ShowMessage(Format('The property %s is %d',[PROPERTY_YEAR,IntegerProperty[PROPERTY_YEAR]]));
  finally
    Free;
  end;
end;

So in this example, we`ve actually declared the property in the const section above deferring the property type until we need it. Its ambiguous so the only reason we need a type is so we know how to store it, and so we can do things to it like add and subtract (numeric types) or truncate and append it (string types). In other words the type isn`t really important until we need it at runtime and until we have to store it.

This will give you some clues into how we are going to achieve persistent properties from one place in our code, without the need to specify property declarations every time we need a new property that is based upon a simple type (ie Integer, Byte, Word, Single, Double, String etc). Using these simple types we can handle all ordinal types like sets.

In the next article, we'll start implementing our property manager class.

Until then, good coding.

Brian Joseph Johns