Hello,
I would like to use the Projection2 component to display a lidar point cloud on top of a video stream.
I get my point cloud from my lidar which is sent to a Projection2 then to an Overlay component.
I can see the result in the image viewer, however the point cloud is not correctly rotated.
I assume I am doing something wrong with the Projection’s matrix input
Can you help me with this ?
Hello,
The Projection2 component was made to use OpenGL projection matrixes.
A quick word on how matrixes work in OpenGL and homogeneous coordinates :
Let’s say you have a point A (x, y, z)
In homogeneous coordinates this become x, y, z, w
This allows all transformations to be done in one matrix i.e :
(x', y', z', w') = [scaling + rotation + shear + translation] × (x, y, z, w)
Examples can be found on the khronos website :
Rotation (linear in 3d) :
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glRotate.xml
Translation (not linear in 3D) :
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glTranslate.xml
LookAt (used to set the camera position in a “world reference frame”) :
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/gluLookAt.xml
Projection2 input matrixes :
The component waits for a 3x4 matrix instead of a the usual 4x4 because we don’t need the z as it is simply the z-index of the point which is irrelevant here.
This gives a good starting point for a projection perspective from camera parameters :
https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/gluPerspective.xml
And finally back to RTMaps, here is a demo which uses the Projection2 component if you want to see it in action :