SOFA plugin with some additional libraries

[bgarcial]

Hi SOFA Community

Currently I want add a new plugin to SOFA.
This is related with the goal of communicate from SOFA some data or parameters such as:
– Position of user instrument in relation with the simulation scenario
– Haptic feedback
– Topological changes …

Initially I want make a concept proof in which from SOFA, I could send some data to another process or external entity, for example the position of the user’s instrument in a surgical simulation scenario.

For this objective I am using the zeroMQ framework which is a concurrent messaging framework to distributed applications, via sockets and some messaging patterns …

Currently, I am setup the project structure plugin based in the Sensable plugin sample.

To work with zeroMQ, is necessary download and build some libraries with make, such as libsodium and the zeroMQ tarball software package. In this README I describe the process.

According to the guide to create a SOFA plugin, I have this repository in which I have the directory structure according to SOFA require it.

My question is:

How to can I configure the libraries required to zeroMQ inside sofa configuration plugin?

I have been taking like base the src/applications/plugins/Sensable/ and I can construct my own directory structure plugin:

I have a CMakeLists.txt to my plugin in which I reference the source and header files that I am using.
https://github.com/bgarcial/sofa-plugin/blob/master/ZeroMq/CMakeLists.txt

And I have the initZeroMq.cpp initialization file, in which I am describing the plugin and linking the unique Class which I am using.
https://github.com/bgarcial/sofa-plugin/blob/master/ZeroMq/initZeroMq.cpp

This class is named Client() and is defined in my Client.h file

How to can I think in build sofa to create this plugin to integrate to SOFA and work with zeroMQ functionalities inside SOFA core?

I hope my question hasn’t been too long, it’s just that I wanted to give the details of what I’m doing and the objective I want to reach.

Best Regards.

[Hugo]

hi @bgarcial,

Thank you for your interest in SOFA and starting coding to integrate your work!
I would advise to get inspiration from the LeapMotion plugin (instead of Sensable).
Moreover you can find some documentation online on how to create a plugin.
Let us know if it works or if you have some errors, we would help you.

Cheers,

Hugo

[bgarcial]

Hi @hugo

In first instance thanks for your answer.

Now, I am referencing the LeapMotion that you tell me.
Next, I place my plugin directory in sofa/src/applications/plugins/

Currently, I configure (via cmake-gui) and build the plugin which I am generating, and I get this error:

Apparently, my C++ application which I want work inside of SOFA, does not recognize the zeroMQ data type socket.

My doubt or inconvenient is how to add or configure the zeroMQ library which I am calling here (#include <zmq.hpp>)

I unknown how to setup the external libraries which my plugin will use.
In my CMakeLists.txt may be?

[bgarcial]

Hi @hugo I think the key is in this file in the LeapMotion plugin sample to call the libraries that I need

According to you says me, I will watch in a deep way the LeapMotion structure directory and their properties to construct my plugin.
I appreciate your comments 🙂

[Hugo]

alright @bgarcial,

Let us know if you get stuck or if you solve the issue !
Cheers,

Hugo

[ErwanDouaille]

Hi @bgarcial,

We (Defrost team) were working on something close to your need.
I just created a PR available on : https://github.com/sofa-framework/sofa/pull/534

Let us know if it fits your needs. I started to write a documentation : https://github.com/SofaDefrost/sofa/tree/sofaCommunication/applications/plugins/Communication

Actually there is a lack of matrix support but you can send data such as float/double … there is also some examples in the project directory.

[bgarcial]

Hi @erwandouaille perfect.

I’ll start taking a look at the plugin you’ve built.

I see that in this file they call the libraries, among them Izmq
https://github.com/SofaDefrost/sofa/blob/sofaCommunication/applications/plugins/Communication/CMakeLists.txt#L34

I see you have worked the plugin so that from a scene can send and receive data from it via zeroMQ, is that correct?

In this sense, for the moment, one of ours objectives is send the position of user instrument in relation with the simulation scenario.
Initially, I am configuring the zeroMQ library, and our task after, will be setup the communication of scene data via zeroMQ sockets. In this sense, I have a question which can seem obvious or not and also more according to our needs …

Can I reuse the SofaCommunication plugin in relation to zeroMQ functionalities and not reinvent the wheel in relation to configurations libraries and implement communication data?

I know that this question is more to analyze by myself in more detail too .

Best Regards for share the product and knowledge.

[ErwanDouaille]

I see that in this file they call the libraries, among them Izmq
https://github.com/SofaDefrost/sofa/blob/sofaCommunication/applications/plugins/Communication/CMakeLists.txt#L34

Exactly, it´s maybe not the best cmake code ever. But it´s still a WIP project. I only tested it on linux.

I see you have worked the plugin so that from a scene can send and receive data from it via zeroMQ, is that correct?

Correct ! The main objective is to be more user friendly and add the ability to easily add communication´s part in the scenes. You can plug in it (subscribe) to differents data´s component.

Give a try to the examples, quite fast to use. There is cpp files associated to sofa´s scene. By default the communication plugin is turned on by cmake.

In this sense, for the moment, one of ours objectives is send the position of user instrument in relation with the simulation scenario.
Initially, I am configuring the zeroMQ library, and our task after, will be setup the communication of scene data via zeroMQ sockets. In this sense, I have a question which can seem obvious or not and also more according to our needs …

Can I reuse the SofaCommunication plugin in relation to zeroMQ functionalities and not reinvent the wheel in relation to configurations libraries and implement communication data?

In my opinion yes. The limitation is if your datas are matrix. (I didn´t find a correct sofa type for it).
What kind of data would you like to send ? Any speed requirements/limitations ?

One more thing. This plugin is not sync with the sofa thread. Meanings you can receive a lot of datas between each steps of sofa.

[bgarcial]

Hi @erwandouaille

Definitely I have to test the SOFA Communication Plugin, I wanted to share with you some concerns about it:

Give a try to the examples, quite fast to use. There is cpp files associated to sofa´s scene. By default the communication plugin is turned on by cmake.

I want test the Communications plugin samples, How to can I do it?
I mean, in the README.md says some considerations.

In relation to them … I have my own local SOFA instance. I will download the SOFA plugin Communications structure directory and I will add in my sofa instance (/applications/plugins/…) and compile it.

Some additional considerations to test some basic samples in which I can test send/receive data from a SOFA scene?

My apologies if I request too much to you (this is not intention), is that the documentation is not clear for me. Sorry. However this documentation whatever give great notions to follow in the process.

About of reuse the Communications Plugin, you says:

In my opinion yes. The limitation is if your datas are matrix. (I didn´t find a correct sofa type for it).
What kind of data would you like to send ? Any speed requirements/limitations ?

Initially we want send from SOFA the following data characterization:

– Data about of instrument position
In this sense I will think that floats and/or doubles data type is suited …

– Data about of collision of instrument user with the simulation scenario.
In this sense, we send information in some vector/array in three dimensions according to x,y,z axes in relation to 3D object models nature.

– Haptic feedback
This feedback too with vector/array in three dimensions

– Topological changes
Arrays of int type, in relation to id’s of each change in the mesh, in where each id is mapped to x,y,z axes …

However, beyond all this I want test the communication plugin to know more about it, and can think in use and why not collaborate just in case …

[ErwanDouaille]

I want test the Communications plugin samples, How to can I do it?
I mean, in the README.md says some considerations.

In relation to them … I have my own local SOFA instance. I will download the SOFA plugin Communications structure directory and I will add in my sofa instance (/applications/plugins/…) and compile it.

Some additional considerations to test some basic samples in which I can test send/receive data from a SOFA scene?

The samples may works well. Ensure you get the good versions of libs. As I already said, I never test it on windows. So maybe you will have to change some cmake stuff. If so, let us know. We will grant you write access to the communication branch.

In the samples there is some cpp files. You just need to compile it to make the related scene works. Or you can send data by yourself on the scene´s protocol port.

– Data about of instrument position
In this sense I will think that floats and/or doubles data type is suited …

Works

– Data about of collision of instrument user with the simulation scenario.
In this sense, we send information in some vector/array in three dimensions according to x,y,z axes in relation to 3D object models nature.

Works. Actually a vector(should be matrix). Or you can send a non matrix message with x y z parameters and store it in corresponding x y z component parameters

– Haptic feedback
This feedback too with vector/array in three dimensions

Need the matrix support

– Topological changes
Arrays of int type, in relation to id’s of each change in the mesh, in where each id is mapped to x,y,z axes …

Same as collision IMO.

However, beyond all this I want test the communication plugin to know more about it, and can think in use and why not collaborate just in case …

Sure, it will be great for the communication plugin. The more we will be involved in it, better it will be. If you need some live help let ne know.

[ErwanDouaille]

Hi @bgarcial,

I just did some work around matrix support. Did you try the communication plugin or make progress on your zmq imlpementation ?

[bgarcial]

Hi @erwandouaille

Currently I am customize my zmq implementation with some classes and structs to receive some data from sofa about of collisions and deformation calculus

I cannot test the communication plugin in the recent time.

Can you tell me about of the latest commits in relation to matrix support.
I can see that these changes are integrated in sofa … is this really?
https://github.com/sofa-framework/sofa/commit/2d147c2

Best Regards

[ErwanDouaille]

Hey,

There is some discussion about the communication plugin and it might be include for the next release as an experimental feature.

For the matrix support, as few as i tested it works well.
What kind of information do you need ? 🙂

[bgarcial]

Hi @erwandouaille

There is some discussion about the communication plugin and it might be include for the next release as an experimental feature.

This sounds great!

Initially we want send from SOFA the following data characterization:

– Data about of instrument position
In this sense I will think that floats and/or doubles data type is suited …

– Data about of collision of instrument user with the simulation scenario.
In this sense, we send information in some vector/array in three dimensions according to x,y,z axes in relation to 3D object models nature.

– Haptic feedback
This feedback too with vector/array in three dimensions

– Topological changes
Arrays of int type, in relation to id’s of each change in the mesh, in where each id is mapped to x,y,z axes …

I know that this questions I already made to you … But, the documentation about of how to use Communication plugin is not clear for me … Can I help in detail the documentation if you want, although is possible that the inconvenient may be just of me and nothing more …

I would like can test the Communication plugin, is just that there is somethings (after of the installation) which I haven’t clear for the moment.

[ErwanDouaille]

Initially we want send from SOFA the following data characterization:

– Data about of instrument position
In this sense I will think that floats and/or doubles data type is suited …

– Data about of collision of instrument user with the simulation scenario.
In this sense, we send information in some vector/array in three dimensions according to x,y,z axes in relation to 3D object models nature.

– Haptic feedback
This feedback too with vector/array in three dimensions

– Topological changes
Arrays of int type, in relation to id’s of each change in the mesh, in where each id is mapped to x,y,z axes …

We did it exactly for the same reasons/objectives.

I know that this questions I already made to you … But, the documentation about of how to use Communication plugin is not clear for me … Can I help in detail the documentation if you want, although is possible that the inconvenient may be just of me and nothing more …

I would like can test the Communication plugin, is just that there is somethings (after of the installation) which I haven’t clear for the moment.

Ok I will be glad to help you, and give github access to the repository as well. Can we organize a skype or whatever call to help you in the communication installation/test/question … ?

I will be available this monday and tuesday. UTC time ?

Thanks for helping

[bgarcial]

Hi @erwandouaille

In this moment we have some C++ structures which represent some events which happen in the surgical scenario such as cutting, carving, probing and attaching …

This structs and data members are here:

As you can see, in this structs we have different data types such as vectors, strings, Quat, between others)

With the communication plugin (this is something that you has been talk to me in all previous messages) we have send these different struct data types from SOFA? Even … We can send struct objects?

It’s great that you can support me in the install and test process of Communications Plugin. I was slow to respond to this message, but if you want and have availability we can interact via Skype the Tuesday or any day and try perform the installation and test the use.

Best Regards

[ErwanDouaille]

Hey,

With the communication plugin (this is something that you has been talk to me in all previous messages) we have send these different struct data types from SOFA? Even … We can send struct objects?

Actually no. The communication plugin is design to send sofa’s data of components. In your case :
Data<double> myparam;
Data<std::string>d_address;

But we can create on runtime the equivalent of your struct as datas. I guess … We need to test it.

It’s great that you can support me in the install and test process of Communications Plugin. I was slow to respond to this message, but if you want and have availability we can interact via Skype the Tuesday or any day and try perform the installation and test the use.

Your welcome, the more we share/use/critics the communication plugin, better it will be.

Ok, for tuesday 19 3pm UTC ? Hangout (douailleerwan[at]gmail.com) ?

[bgarcial]

Hi @erwandouaille

Actually no. The communication plugin is design to send sofa’s data of components. In your case :
Data<double> myparam;
Data<std::string>d_address;

I have the following concern about of Data type component:
If one component have many Data, is possible concatenate them and send them in only request send? This is of a selective way in one request ZMQ communication socket operation?

Or

When the send request is performed, all the Data component are sent without select which of them I would like sent?

I say or ask this, because in our structures, some member data will be component data, but it is not in our interest to send all component data because our application is network communication and it is necessary to take into account performance, and not send irrelevant data.

In this basic diagram I will want draw the workflow

Your welcome, the more we share/use/critics the communication plugin, better it will be.

Ok, for tuesday 19 3pm UTC ? Hangout (douailleerwan[at]gmail.com) ?

I appreciate your availability …
This will be great. Just a small inconvenient ..
My english language (listening skills) are basic …
Is possible combine the possibility of chat and talk about it?

I will be pending.

[ErwanDouaille]

I have the following concern about of Data type component:
If one component have many Data, is possible concatenate them and send them in only request send? This is of a selective way in one request ZMQ communication socket operation?

Or

When the send request is performed, all the Data component are sent without select which of them I would like sent?

You have to select which data you have to send. I will show you this afternoon, it´s easy, and it´s the way we design the plugin.

In this basic diagram I will want draw the workflow

My team is also doing some communication between different sofa, that´s the reason why we design this plugin.

I appreciate your availability …
This will be great. Just a small inconvenient ..
My english language (listening skills) are basic …
Is possible combine the possibility of chat and talk about it?

Same for me, don´t worry 😉

[ErwanDouaille]

@bgarcial just to let you know than I just fixed the zmq problem we had during the visio.

Let me know if you succeed compiling it and if it works

[bgarcial]

Hi @erwandouaille, I appreciate your orientation and support in the Communication Plugin.
Recent I back to activities and I bring apologize to you for my late response.

For the moment I am testing the data communications from SOFA zeromq client to External Server python of a manual way, with the order to understand some implications in relation to zmq working with sofa data and structures, like I was comment to you in previous opportunity …

In this situation, I have been dealing with the way of how to the destination entity interpret or receive some message / data and this is showed or deployed.

In the Communication plugin … how to manage you the serialization when receive messages independent of the platform or language?

This is related with the following question and the answer provided by the community.

I hope this is not an inconvenience for you 🙂

[Hugo]

Hi @bgarcial

I hope your work with the Communication plugin are successful. I would be glad to hear more about your ongoing work. Let us know if we can help in any way.

Best

Hugo

[Author]

Posts