inria rennes how to cookopenvibe.inria.fr/openvibe/wp-content/uploads/2016/06/jl... ·...
TRANSCRIPT
1
J. T. Lindgren / OpenViBE 1
Jussi T. Lindgren, [email protected]
Inria Rennes
HOW TO COOK
C++ BOXES
OpenViBE workshop
Asilomar, June 2016
2
J. T. Lindgren / OpenViBE 2J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Boxes?
C++ boxes, anyone?
Inside your C++ box
Finally an example
Everything is not a box
Contents
3
J. T. Lindgren / OpenViBE 3J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Boxes?
4
J. T. Lindgren / OpenViBE 4J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Recall: signal processing is
done by boxes
Signal processing boxes
included in OpenViBE...
...potentially
allow you to do
1001 things already
OpenViBE Designer
5
J. T. Lindgren / OpenViBE 5J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
But you want to do thing #1002...
• Try to use existing boxes in a clever way
• Make a wish
• Hire an engineer
• Custom order from a company...
• Implement a new box yourself
6
J. T. Lindgren / OpenViBE 6J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Whats a box?
A box can be
• A wrapper that actually runs Matlab, Python or Lua
script to do its thing (David’s talk)
• A C++ class (this talk)
7
J. T. Lindgren / OpenViBE 7J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
View 1. Box as a function out = f(in)
Input streams(s)
Output streams(s)
Box
Box::process()
{
// get input
// do stuff
// send output
}
A good start!
8
J. T. Lindgren / OpenViBE 8J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
View 2. Box as a processor
• OpenViBE – a streaming
architecture
• Boxes process data in chunks
• Data chunks after
elaborate transformation
4
3
2
…
1
0
Head
Tim
e pas
ses
Signal
chunks
9
J. T. Lindgren / OpenViBE 9J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
View 3. A programmer’s box
Box {• Description
• User parameters declared
• Reactions to parameter changes
• Input/output handling if any
• Your algorithm
}
• A C++ box is a class with several member functions
• It needs 30+ lines of ’glue code’ !
10
J. T. Lindgren / OpenViBE 10J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
C++ boxes, anyone?
11
J. T. Lindgren / OpenViBE 11J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Why C++ boxes?
Pros• C++ is the implementation language of OpenViBE
• No external dependencies needed
• Code can be as fast as you can make it
• No added python/matlab compatibility issues
Cons
• Needs a compiler (free ones are ok)
• Modifications require compilation and restart
• C++ prototyping can be slow
• Getting started with C++
http://openvibe.inria.fr/build-instructions
12
J. T. Lindgren / OpenViBE 12J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Approach 1. Modify an old C++ box
Simple:
• Select an old box that resembles your new box
• Locate the box code from the source tree (’plugins/’)
• The box is usually a .h / .cpp pair
• Hack away
Your starting point has:
• Some ’interface glue’ already in place and it works
• Some old extra stuff, lacks your new stuff
13
J. T. Lindgren / OpenViBE 13J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Approach 2. Skeleton Generator tool
• Skeleton Generator gives you ’the bare bones’
• Creates a pre-filled .cpp/.h pair
• You add your meat custom processing code
14
J. T. Lindgren / OpenViBE 14J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Specify all the
properties of your box
When done:
Check, then
Generate!
SKELETON
GENERATOR
15
J. T. Lindgren / OpenViBE 15J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Skeleton Generator tutorial
http://openvibe.inria.fr/tutorial-1-implementing-a-signal-
processing-box
16
J. T. Lindgren / OpenViBE 16J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Inside your C++ box
17
J. T. Lindgren / OpenViBE 17J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box description
• Each box needs a description
• Box identifier (random 64bit number)
• Box name
• Author
• Inputs & outputs
• Declaration of box settings
• ...
• These are in the box header .h file
• Used to visualize the box to the user
• The header may also define ’listeners’ : code that reacts to box
parameter changes that the user does in Designer
18
J. T. Lindgren / OpenViBE 18J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box settings
Settings declared in the .h
file get a generated GUI
Box settings are saved/loaded
with the scenario automagically
Getting value of an uint64 param at slot 3 (index from 0):
uint64 l_ui64ThirdSettingValue
= FSettingValueAutoCast(*this->getBoxAlgorithmContext(),2);
19
J. T. Lindgren / OpenViBE 19J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
1
0
Codecs
• Boxes have codecs
• Like audio/video codecs
• Codecs marshall data
chunks to/from streams
• Each stream needs its
own codec
Decode is required to
get at the raw data!
4
3
2
Stream
EBML
encoded
EBML
encoded
Decoder
Encoder Box
20
J. T. Lindgren / OpenViBE 20J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Boxes follow stream conventions
Header
Chunk 0
Chunk 1
Chunk n
End
…
TimeDefines the stream properties, e.g.
content type, chunk size (if any),
sampling rate, etc.
Actual data. Box may have several
chunks pending for process()
Just an event, no content
Stream
t=0
t=0
t=1
…
t=n
t=n
21
J. T. Lindgren / OpenViBE 21J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
What time is it??
• Box code may often rely on the time of a block or sample
(e.g. ”process data for 2 secs from time t of stimulation”)
• OpenViBE uses 64bit fixed-point time• 32bits for seconds, 32bits for subseconds
• Equally precise both for ’small’ and ’large’ numbers
• class ITimeArithmetics has some convenience functions • 64bit fixed point time float64 seconds
• 64bit fixed point time sample count
• Seconds + subseconds pair 64bit fixed point
22
J. T. Lindgren / OpenViBE 22J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Finally an example
23
J. T. Lindgren / OpenViBE 23J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
A signal modifying box
Box::Process() {
// Decode;
// Modify;
// Encode;
}
Signal chunks in
Modified signal chunks out
Box::Init() {
// read GUI
// settings;
// Init codecs;
}
Box::Uninit() {
// free
// resources;
}
24
J. T. Lindgren / OpenViBE 24J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box header (pseudocode)
TSignalDecoder< myBox > m_signalDecoder;
TSignalEncoder< myBox > m_signalEncoder;
class myBoxDesc {
getName(void) { return ”MyBox"; }
getAuthorName(void) { return ”Somebody”; }
getAuthorCompanyName(void) { return ”ACME”; }
// ...
getBoxPrototype(OpenViBE::Kernel::IBoxProto& rPrototype)
{
rPrototype.addSetting(”GUI setting 1",
OV_TypeId_Float, "1.0");
rPrototype.addInput("Input 1", OV_TypeId_Signal);
rPrototype.addOutput(”Output 1", OV_TypeId_Signal);
}
}
25
J. T. Lindgren / OpenViBE 25J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box initialization
myBox::initialize() {
m_signalDecoder.initialize(*this, 0); // In-connector 0
m_signalEncoder.initialize(*this, 0); // Out-connector 0
// make decoder and encoder share buffer & sampling rate
m_signalEncoder.getInputMatrix().
setReferenceTarget(m_signalDecoder.getOutputMatrix());
m_signalEncoder.getInputSamplingRate().
setReferenceTarget(
m_signalDecoder.getOutputSamplingRate());
}
26
J. T. Lindgren / OpenViBE 26J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box processing
myBox::process() {
for(uint32 i=0; i<nChunks; i++) {
m_signalDecoder.decode(i); // decode chunk i
if(m_signalDecoder.isHeaderReceived()) {
// n.b. input stream properties are now
// available via m_signalDecoder getters …
m_signalEncoder.encodeHeader();
}
if(m_signalDecoder.isBufferReceived()) {
float64* l_pData
= m_signalDecoder.getOutputMatrix()->getBuffer();
// ... manipulate l_pData here ...
m_signalEncoder.encodeBuffer(); // encode chunk
}
l_rDynamicBoxContext.markOutputAsReadyToSend(0 , ...);
}
}
27
J. T. Lindgren / OpenViBE 27J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Box uninitialization
myBox::uninitialize() {
// Free resources
m_signalEncoder.uninitialize();
m_signalDecoder.uninitialize();
}
That concludes a box.
28
J. T. Lindgren / OpenViBE 28J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Everything is not a box
29
J. T. Lindgren / OpenViBE 29J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
• Amplifier driver is not a box• Make it an Acquisition Server component instead
• Algorithm is perhaps not a box• If a routine is shared by several boxes, consider making it a class or an
’Algorithm’
• Stimulators may be external apps if they like• E.g. separate SSVEP, P300 flashers ...
• Right after rendering, send the event markers
to Acquisition Server using ”TCP Tagging”
30
J. T. Lindgren / OpenViBE 30J.T. Lindgren: OpenViBE C++ boxes OV Workshop Jun 2016
Here we have skipped a lot of details:
Box coding is best learned by doing
More about at
http://openvibe.inria.fr
That’s it!