NEW: Kinect Support - Cinder Forums

Loading...

Cinder Forums

Recent Topics
All Forums
- Search All
- Tags
- Author
- Advanced Search
Search
New Topic

Developing Cinder

NEW: Kinect Support

in Developing Cinder • 2 years ago

We're happy to announce support for the Kinect in Cinder under Mac OS X. It's available here:

https://github.com/cinder/Cinder-Kinect

To get started, be sure you've updated and rebuilt Cinder from the current github repository. Please note that this does not compile with 0.8.2 - you'll need the latest github repo. After updating Cinder, you should:

cd <path to cinder>/blocks

git clone git://github.com/cinder/Cinder-Kinect.git

The CinderBlock comes with a sample that should work out of the box. If you create a new project that uses this CinderBlock, note that it requires the Mac OS X framework IOKit.framework as well as the static libraries libusb-1.0.a and libfreenect.a, and the source file Kinect.cpp.

Here's a sample of what Robert Hodgin created using this CinderBlock: http://vimeo.com/17075378

Special thanks to Cinder user Rui Madeira, who's original code is what jumpstarted this effort.

3

Replies(52)

Andrea

Re: NEW: Kinect Support

2 years ago

that was fast :]

I just bought a Kinect on amazon and I can't wait to get it working with Cinder.

Re: NEW: Kinect Support

2 years ago

Robert has written a new sample which is in the repo called kinectPointCloud.

Kinect in Cinder, Hello World from flight404 on Vimeo.

Additionally, we've tested Cinder-Kinect with two Kinects and it appears to work (minus the potential for the IR patterns interfering with each other).

pixelnerve

Re: NEW: Kinect Support

2 years ago

sweet.
i have a clnui wrapper coming on. would like to know how about having 2 kinect blocks?
the above one runs on top of libfreenect and the other on clnui.
clnui only works for windows systems, and afaik freenect is still mac only?!

section9

Re: NEW: Kinect Support

2 years ago

Goddamnit! >< You guys are too fast! Spent ages yesterday sorting this out on my own and now you've finished it :P

But....good work :)

Re: NEW: Kinect Support

2 years ago

Hi ( new here, so please excuse....)

great, support for two devices ! That´s exactly what I´m looking for.

Is there already some documention? I've got it to detect 2 devices (message to console) but not managed to give some commands to both plugged devices. Allways the first plugged in Kinect is talked to.

I have set:

mKinect = Kinect( Kinect::Device(1) );

nKinect = Kinect( Kinect::Device(2) );

Do I have to set different device parameters?

I also noticed that the Kinect stream (using kinectBasicApp) tends to freeze (also with only one Kinect) here. Is there a way to reduce the stream's framerate?

Well, let me say that finding your nice package was one of the nice great finds in these exciting last Kinect days.

Re: NEW: Kinect Support

2 years ago

Hey - the indexing starts with 0, so you should probably do

mKinect1 = Kinect( Kinect::Device(0) );

mKinect2 = Kinect( Kinect::Device(1) );

I haven't seen any slow down yet - do you have a large number of devices on the same USB bus? You might try just having the one plugged in and see if that affects anything in terms of performance...

-Andrew

Re: NEW: Kinect Support

2 years ago

ok this is probably not Kinect code specific but I have trouble with getting the depth image into a surface. it comes back in 16-bit grayscale (as opposed to the color image which is 8 bit RGB). so i'm doing this:

Surface depthSurf = Surface(ImageSourceRef(mDepthTexture));
Surface colorSurf = Surface(ImageSourceRef(mColorTexture));

to be able to access the image data (outside of shaders as in the example). works fine with the colorSurf, but the depthSurf gives me only zeros in its data. how do i create a valid Surface object from the grayscale/16bit/Y-channel-order imageSource?

Re: NEW: Kinect Support

2 years ago

Hey - fastest route is probably to make a Surface directly out of the ImageSource that Kinect returns.

Surface8u depth( mKinect.getDepthImage() );

or you can do

Surface32f depth( mKinect.getDepthImage() );

if you want the data as floating point. If you want to keep it as 16-bit data, Cinder doesn't have a native image type which is 16-bit but you use getDepthData() instead.

shared_ptr<uint16_t> depthData = mKinect.getDepthData();

uint16_t col3Row2Depth = depthData.get()[2*640+3];

-Andrew

dazzid

Re: NEW: Kinect Support

2 years ago

Hi

First thanks a lot for all this amazing info!

I have followed the steps that Andrew suggest. Now I have the cinder_master folder and I've downloaded the last version of boost.

And then when I try to compile the KinectPointCloud Xcode is giving me this error:

Command /Developer/usr/bin/g++-4.2 failed with exit code 1

I'm using Xcode 3.2.4 with Mac Os x 10.6.5

Thanks for the help

David

Re: NEW: Kinect Support

2 years ago

Hi - Have you rebuilt Cinder itself, per the instructions here? Does samples/basicApp build for you?

-Andrew

dazzid

Re: NEW: Kinect Support

2 years ago

Hi Andrew,

I repeated again the rebuild, drinking this time the coffee and it's working!!!

I supposed last time I did it for the 0.8.2 version, my mystique.

Thanks a lot for the help

Cinder is amazing!

All the best

David

Re: NEW: Kinect Support

2 years ago

Hi. my name is Fumi.

I have one question.
I couldn't run cinder Kinect yet.

I put the yjtyjtyj on blocks, but it doesnt work ;)
Could you tell me !
Does anybody know any ideas ??

thanks,

Fumi from sendai,Japan

dazzid

Re: NEW: Kinect Support

2 years ago

Hi Fumi,

Read this link carefully. When the rebuilt is done you will have Cinder running well.

http://libcinder.org/docs/welcome/GitSetup.html

Re: NEW: Kinect Support

2 years ago

Hey Fumi,

Your errors show that you have an older version of Cinder (0.8.1). You need to have the latest version of Cinder to get the latest features.

The link Dazzid posted will help you get the right version of Cinder to have Kinect support. Most of the time, when features are announced on the forum, those features are currently available in the latest github repository. The downloaded versions of Cinder are static once you grab them.

Re: NEW: Kinect Support

2 years ago

Any chance to get this working on win7?

I guess not to the IOKit framework :/

Re: NEW: Kinect Support

2 years ago

I'm with Sorny... Just went out and bought a kinect tonight, but i'd love to play around with cinder and kinect on windows.

Really looking forward to messing about with this!
Even

Re: NEW: Kinect Support

2 years ago

Deer
dazzid and sansumbrella

Thank you very much for your time and help.
My problems was the C++ boost and current Cinder version.

fumi

pixelnerve

Re: NEW: Kinect Support

2 years ago

Steven, i wrote a wrapper for CLNUI to work with Cinder.. you can download it at:
http://dl.dropbox.com/u/4671020/ExampleCLNUI.zip
It not updated as the java version but its enough to get one started. It's based on the actual kinect block.
For this one you need drivers and SDK from http://www.codelaboratories.com/nui

In case you really want to go OpenKinect then look at this page. its OpenKinect (libusb) for Windows:
http://ajaxorg.posterous.com/kinect-driver-for-windows-prototype

It shouldn't take long to get current addon running on windows, just needs someone to do it

marcinignac

Re: NEW: Kinect Support

2 years ago

I have some problems with creating new Kinect project using TinderBox for XCode. I added libraries (libfreenect.a, libusb-1.0.a) to Frameworks group but I keep getting linker errors. I checked example project and I can't find where you added them. Any hints?

Re: NEW: Kinect Support

2 years ago

Hey - what are your link errors? In general Cinder projects link using the Other Linker Flags of the Target, because that is the most consistent way to get proper static linking.

marcinignac

Re: NEW: Kinect Support

2 years ago

The problem was missing IOKit.framework. I found the solution here:

http://forum.libcinder.org/topic/simple-hand-tracking-with-kinect-opencv

Re: NEW: Kinect Support

2 years ago

Hey there,

I'm running into a problem trying to transform the depth values into metres, using the Point Cloud example as a jumping-off point.

In the vertex shader, the sample seems to take the blue value of the depth-texture pixel (0.0f - 1.0f) and uses a factor of 1000 to distort the mesh.

http://graphics.stanford.edu/~mdfisher/Kinect.html discusses the fact that the depth values have an inverse linear relationship to actual depth, and gives coefficients and a formula for converting depth values in the range 0-2048 to metres, using the following formula:

1.0 / (double(depthValue) * -0.0030711016 + 3.3309495161)

When I try and incorporate that into the example, I get a nearly flat mesh. I'm using

vVertex.z += 1.0 / (depth * 2048.0) * -0.0030711016 + 3.3309495161);

Anyone else trying to get the depth values mapped to real-world measurements?

Re: NEW: Kinect Support

2 years ago

Hey - if you checkout the source for the Cinder block, you can see that it's somewhat hacky in that it moves the values directly into a visually sensible space, at the cost of being able to properly reconstruct the depth values.

https://github.com/cinder/Cinder-Kinect/blob/master/src/Kinect.cpp#L204

What should probably happen is we add a method like

Kinect::setDepthNormalize( bool normalize = true ); and then we just don't do that math if you want to get at the raw data. Anyone want to send me a pull request along those lines?

-Andrew

Re: NEW: Kinect Support

2 years ago

Oh, that makes sense. I ended up eyeballing it to find a reasonable depth scaling coefficient. For anyone looking, 2000 seems to do a reasonable job in Robert's Kinect Point Cloud example.

I would love to help but really you don't want me and my pidgin C++ anywhere near the library code. There's a joke there somewhere about the difference between dancers and coders... dancers can count to eight, but have to start at five; coders can only count to one, but have to start at zero?

Re: NEW: Kinect Support

2 years ago

Hi all. Thanks for posting this, it will be very useful. I am however running into an issue where the image from the Kinect seems to freeze nearly as soon as the app starts. Has anyone else experienced this? I am running OSX 10.6 on a Mac Mini 1.66 GHz with 2GB of RAM.

Thanks

Jay

Re: NEW: Kinect Support

2 years ago

Hi Jay,

I've got the same issue as you, with my 2007 Macbook Pro, although I don't get the same problem with code from Open Frameworks' Cinder test code.

I've also tried the Cinder code on a Macbook Air, and it doesn't exhibit this same lockup, so it may well be related to the USB host controller in certain hardware.

Nick

Re: NEW: Kinect Support

2 years ago

hi,
i ve installed xcode 3.2.3 64bit on macbook pro with i5

i have rebuilt Cinder and all but the kinect basic app doesnt run.
this is my error

Command /Developer/usr/bin/g++-4.2 failed with exit code 1

ahhhh.

please help...

greeting from germany
chris.

Suje

Re: NEW: Kinect Support

2 years ago

I was getting that msg when I forgot to run ./fullbuild.sh

Re: NEW: Kinect Support

2 years ago

I am having the same error message.

Command /Developer/usr/bin/g++-4.2 failed with exit code 1

What do you mean by running .fullbuild.sh?

How should I do so?

Could you explain a bit more about it?

Thanks

Suje

Re: NEW: Kinect Support

2 years ago

"when updating from the github to build both the Debug and Release configurations run the script located at cinder/xcode/fullbuild.sh "

cd xcode
./fullbuild.sh

Re: NEW: Kinect Support

1 years ago

Hi guys,

am just trying to get this to work for the first time on a macbook pro. Followed the steps in setup, section by section but getting build errors when i run

./fullbuild.sh

I downloaded cinder and put it in

~/cinder/

installed git clone and ran the git clone command - but i did it from a different directory

~/CinderGIT/

So now the cinder_master dir is

~/CinderGIT/cinder_master/

Then I downloaded the latest version of boost, unarchvied it and renamed resulting directory as "boost". Then copied this to

~/cinder/

which overwrites the one that came with the original download.

Finally I ran ./fullbuild.sh from

~/CinderGIT/cinder_master/xcode/

Lots of building, with the final lines being the following (i didn't post the full output as its hundreds of lines)

Camera.cpp:23:

../include/cinder/Cinder.h:25:29: error: boost/cstdint.hpp: No such file or directory

../include/cinder/Cinder.h:87:71: error: boost/shared_ptr.hpp: No such file or directory

In file included from ../include/cinder/CinderMath.h:25,

from ../include/cinder/Vector.h:36,

from ../include/cinder/Camera.h:25,

from /Users/bobbydazzler/CinderGIT/cinder_master/xcode/../src/cinder/Camera.cpp:23:

../include/cinder/Cinder.h:28: error: 'boost' has not been declared

../include/cinder/Cinder.h:29: error: 'boost' has not been declared

../include/cinder/Cinder.h:30: error: 'boost' has not been declared

../include/cinder/Cinder.h:31: error: 'boost' has not been declared

../include/cinder/Cinder.h:32: error: 'boost' has not been declared

../include/cinder/Cinder.h:33: error: 'boost' has not been declared

../include/cinder/Cinder.h:34: error: 'boost' has not been declared

../include/cinder/Cinder.h:35: error: 'boost' has not been declared

In file included from ../include/cinder/CinderMath.h:25,

from ../include/cinder/Vector.h:36,

from ../include/cinder/Camera.h:25,

from /Users/bobbydazzler/CinderGIT/cinder_master/xcode/../src/cinder/Camera.cpp:23:

../include/cinder/Cinder.h:88: error: 'boost' has not been declared

** BUILD FAILED **

The following build commands failed:

CompileC build/cinder.build/Debug-iphonesimulator/cinder_iphone_sim.build/Objects-normal/i386/Utilities.o ../src/cinder/Utilities.cpp normal i386 objective-c++ com.apple.compilers.llvmgcc42

CompileC build/cinder.build/Debug-iphonesimulator/cinder_iphone_sim.build/Objects-normal/i386/Camera.o ../src/cinder/Camera.cpp normal i386 c++ com.apple.compilers.llvmgcc42

(2 failures)

Any ideas anyone? The only thing i can think of is maybe i should have cinder in my home directory, or that the git clone folder isn't inside the cinder directory?

Thanks

Re: NEW: Kinect Support

1 years ago

Sorry stupid mistake, boost has to go into the cinder_master folder. Works now.

Re: NEW: Kinect Support

1 years ago

Almost there, am getting this error when i try and build either of the kinect sample projects.

i686-apple-darwin10-llvm-g++-4.2: ~/CinderGIT/cinder_master/lib/libcinder_d.a: No such file or directory

Command /Developer/usr/bin/llvm-g++-4.2 failed with exit code 1

By default i open up the project and get loads of errors until i change the CINDER_PATH to my rebuilt cinder location "~/CinderGIT/cinder_master". This fixes all but this last error, which is weird as the libcinder_d.a file it can't find is definately in that location.

by the way this is being built on a macbook pro with osx 10.6.7 and xcode 4.0.2

Any ideas?

Thanks

Re: NEW: Kinect Support

1 years ago

Figured it out, think it was the fact that i was using ~/ for home directory rather than the absolute /users/<USR>/

All good

libFreenect with cinder

1 years ago

Hi Everyone,

I used libfreenect to read the data from Kinect and now working on the image processing with OpenCV and maybe later with PCL. how can i upload the code that i had until now so others can use it for Connecting Cinder to Kinect on Windows with Libfreenect.

Thanks,
Hadi

Re: NEW: Kinect Support

1 years ago

Hi, I'm new to Cinder. I'm doing a projection mapping project that I have working on openframeworks and I want to test it on cinder.

I'm trying to get the kinect block examples running. everything works, compiles and run, but nothing happens on screen and the console gives me this runtime error everyframe.

2011-06-16 20:32:57.407 kinectBasic[10059:a0f] invalid memory allocation

There are 1 Kinects connected.

2011-06-16 20:32:59.624 kinectBasic[10059:a0f] invalid memory allocation

2011-06-16 20:32:59.662 kinectBasic[10059:a0f] invalid memory allocation

2011-06-16 20:32:59.695 kinectBasic[10059:a0f] invalid memory allocation

2011-06-16 20:32:59.711 kinectBasic[10059:a0f] invalid memory allocation

it happens with both kinectBasic and kinectPointCloudApp examples..

any clue?

thanks from Argentina!

Re: NEW: Kinect Support

1 years ago

Hi Chparsons,

The Cinder blocks implemented for MAC OS as it mentioned in the forum. I don't know what exactly you want to do but If you have problems with the blocks... you can easily read the data by Libfreenect and then for the pointcloud use PCL Library...

If you need the libfreenect and cinder codes i can send you mine cause it's working but it's for widows...

Bests,
Hadi

Re: NEW: Kinect Support

1 years ago

Hi, thanks for your reply. I need to get the raw depth data. AFAIK the Kinect Block wraps Libfreenect. That's fine. But I'm getting the invalid memory allocation runtime error......

Re: NEW: Kinect Support

1 year ago

When I try to run the KinectApp sample, I'm getting these errors: http://forum.libcinder.org/topic/cinder-istream-ambiguous-symbol. However, I haven't been able to track down where namespace declarations might be off.

Any idea why?

MSVS10, cinder 8.3

1st part of error msg:

1>------ Build started: Project: KinectApp, Configuration: Debug Win32 ------

1> C:\Program Files\Microsoft SDKs\\Kinect\v1.0 Beta2\Assemblies\Microsoft.Research.Kinect.dll

1> 1 File(s) copied

1> KinectApp.cpp

1>c:\program files (x86)\microsoft sdks\windows\v7.0a\include\objidl.h(173): warning C4099: 'cinder::IStream' : type name first seen using 'class' now seen using 'struct'

1> c:\prj\cinder\include\cinder\stream.h(108) : see declaration of 'cinder::IStream'

1>c:\program files (x86)\microsoft sdks\windows\v7.0a\include\objidl.h(708): error C2872: 'IStream' : ambiguous symbol

1> could be 'c:\program files (x86)\microsoft sdks\windows\v7.0a\include\objidl.h(173) : cinder::IStream IStream'

1> or 'c:\prj\cinder\include\cinder\stream.h(108) : cinder::IStream'

ahoeben

Re: NEW: Kinect Support

1 year ago

I don't think the Kinect block is compatible with windows:
http://forum.libcinder.org/topic/kinect-with-vc2010#23286000000661069

Re: NEW: Kinect Support

1 year ago

if you are using the Cinder kinect block Implemented by Bantherewind:
That Block is using microsoft Kinect SDK, the version of the block that you are using is an old version because after microsoft released it's new beta version there isnt any dll called microsoft.research that's from the older version of microsoft SDK...
I recommend downloading the latest version of the SDK from www.Bantherewind.com and then try it again...

Re: NEW: Kinect Support

1 year ago

Thanks.

I was definitely using the latest version of the kinect SDK and the most recent BanTheRewind I could find. So I don't know what my problem was, though I was probably using out of date _something_ without knowing.

After some driver juggling, I got the pixelNerve code to work. https://github.com/pixelnerve/BlockOpenNI

I am excited to see the new BanTheRewind release though.

Re: NEW: Kinect Support

1 year ago

Have a look at section 3 of this document...
http://www.microsoft.com/en-us/kinectforwindows/download/readme.htm

Good luck.

Re: NEW: Kinect Support

1 year ago

OK, in light of the impending banTheRewind update, I started playing around with this again. I cleaned my MS SDK install and re-installed. I checked my cinder files to make sure those are the most recent as well, but am still getting the aforementioned errors. I'm running Win 7 on MacBook with MSVS10.

The Kinect SDK samples all compile and run great.

The cinder block sample *.exe all run great, but the projects won't compile.

Any ideas?

Re: NEW: Kinect Support

1 year ago

I finally had a chance to get back to this.

So the issue was in order of the header includes!

in Kinect.h, files are included in the order:

#include <cinder/app/App.h>

#include <cinder/Cinder.h>

#include <cinder/Thread.h>

#include <cinder/Utilities.h>

#include <map>

#include <ole2.h>

#include <MSR_NuiApi.h>

#include <vector>

App.h and ole2 weren't playing well together in that order. Once I changed the order, putting App.h last ala:

#include <cinder/Cinder.h>

#include <cinder/Thread.h>

#include <cinder/Utilities.h>

#include <map>

#include <ole2.h>

#include <MSR_NuiApi.h>

#include <vector>

#include <cinder/app/App.h>

everything works great.

now it's fun time.

letsgooutside

Re: NEW: Kinect Support

1 year ago

There is an update on the way very soon, as well. Better layout, improvement in performance and stability, and some very nice new sample applications. :)

http://www.BanTheRewind.com

Re: NEW: Kinect Support

1 year ago

I would like to add colors to the pointcloud's VBO mesh(using GL_points) either video image or depth images textures, how can I achieve this? I simply binded the texture with the triangle mesh but it doesnt seem effctive with the points one, what should I do?

letsgooutside

Re: NEW: Kinect Support

1 year ago

If you can wait a couple days, I'll be posting the latest version of the KinectSDK, which includes a triangle mesh demo with UV coords and lighting.

Basically, you want to...

Create a VBO mesh of triangles (one vertex per pixel, six indices per vertex)
Set UV coordinates for each vertex
On update, create textures from the depth and color images
In the vertex shader...

Use the depth texture to displace the vertices in the VBO
Calculate the vertex's normal
Pass the UV (texture coordinate) to the fragment shader

In the fragment shader

Sample the color image using the UV
Apply phong shading (highly recommended to avoid a "flat" looking image)

In your draw method

Bind the shader
Bind the textures
Draw the VBO

The lighting is key to give the image some depth. You will undoubtedly need to wrap the color image's texture and apply a UV offset, as the video image rarely maps 1:1 to the depth image.

http://www.BanTheRewind.com

Re: NEW: Kinect Support

1 year ago

i ll wait and see! I see already a few features i was after, thanks for the reply!

Re: NEW: Kinect Support

1 year ago

Is it updated? After a quick research i realise that Kinect SDK doesnt run on mac osx? Am i correct?

letsgooutside

Re: NEW: Kinect Support

1 year ago

That is correct (not OSX, but will run on Mac with Windows, of course). I'm wrapping up the Kinect block updates today. I was updating the Emotiv block ahead of this one this past week.

http://www.BanTheRewind.com

Re: NEW: Kinect Support

1 year ago

What is the best way to reset the texture if it freezes (but not the app), i want to be on the safe side to run my app for a show.

© 2012 ZOHO Corp.
All Rights Reserved.
Privacy
Terms & Conditions
RSS
Powered by