summaryrefslogtreecommitdiffstats
path: root/thesis
diff options
context:
space:
mode:
Diffstat (limited to 'thesis')
-rw-r--r--thesis/thesis.tex401
1 files changed, 219 insertions, 182 deletions
diff --git a/thesis/thesis.tex b/thesis/thesis.tex
index 2b6c4ff..6a163e8 100644
--- a/thesis/thesis.tex
+++ b/thesis/thesis.tex
@@ -44,6 +44,7 @@ parskip=never]{paper}
\newacronym{aap}{AAP}{Ambisonics Amplitude Panning}
\newacronym{adat}{ADAT}{Alesis Digital Audio Tape}
\newacronym{alsa}{ALSA}{Advanced Linux Sound Architecture}
+\newacronym{apf}{APF}{Audio Processing Framework}
\newacronym{api}{API}{Application programming interface}
\newacronym{asdf}{ASDF}{Audio Scene Description Format}
\newacronym{bs}{BS}{Binaural Synthesis}
@@ -78,13 +79,19 @@ parskip=never]{paper}
\newglossaryentry{sclang}{
name={sclang},
description={Name of the SuperCollider programming language and the
- interpreter executable of the SuperCollider programming language.}
+ interpreter executable of the SuperCollider programming language}
}
\newglossaryentry{id}{
name={ID},
- description={A name or number, that identifies an object.},
+ description={A name or number, that identifies an object},
plural=IDs
}
+\newglossaryentry{stdout}{
+ name={stdout},
+ description={The standard output is a stream where a program writes its
+ output data to. This can be a log file or a terminal},
+}
+
\makeglossaries
@@ -346,85 +353,66 @@ parskip=never]{paper}
\cleardoublepage
\section{Implementation}
\label{sec:implementation}
- The \gls{ssr}, due to its diverse set of rendering engines, which are made
- available through an extensible framework, and its relatively clean
- codebase, is a good candidate for future large scale \gls{wfs} setups. These type
- of features are not yet implemented though and will need testing.\\
- Therefore I propose the implementation and testing of said feature, making
- the \gls{ssr} capable of rendering on large scale \gls{wfs} setups with many nodes,
- controlled by a master instance.\\
- The sought implementation is inspired by the architecture of sWONDER, but
- instead of creating many single purpose applications, the master/node
- feature will be made available through flags to the \gls{ssr} executable, when
- starting it. This behavior is already actively harnessed eg.\ for selecting
- one of the several rendering engines.
- \begin{figure}[!htb]
- \centering
- \includegraphics[scale=0.9, trim = 31mm 190mm 24mm 8mm, clip]
- {ssr-networking.pdf}
- \caption{A diagram displaying the \gls{ssr} master/node setup with TCP/IP
- socket connections over network (green lines), audio channels (red
- dashed lines) and \gls{osc} connection (blue dashed line). Machines are
- indicated as red dashed rectangles and connections to audio hardware
- as outputs of \gls{ssr} nodes as black lines below them.}
- \label{fig:ssr-networking}
- \end{figure}
- While the \gls{ssr} already has an internal logic to know which loudspeaker will
- be used for what virtual audio source, this will have to be extended to be
- able to know which renderer node has to render what source on which
- loudspeaker (see Figure~\ref{fig:ssr-networking}).
- To achieve the above features, the \gls{ssr}'s messaging (and thus also settings)
- capabilities have to be extended alongside its internal logic concerning
- the selection of output channels (and the master to node notification
- thereof). To introduce as little redundant code as possible, most likely a
- ``the client knows all'' setup is desirable, in which each node knows about
- the whole setup, but is also set to only serve its own subset of
- loudspeakers in it. This will make sure that the rendering engine remains
- functional also in a small scale \gls{wfs} setup.\\
- The lack of a direct \gls{osc} functionality, as provided by the two other
- renderers, will not be problematic, as master and nodes can communicate
- through their builtin TCP/IP sockets directly and the master can, if
- needed, be controlled via \gls{osc}.
-
- \subsection{Prelimenaries}
- \label{subsec:preliminaries}
- In preparation to this work, an implemention of a side-by-side
- installation to the \gls{os} currently driving the \gls{wfs} system setup
- of the Electronic Studio at TU Berlin
- \citep{website:tu-electronic_studio} was attempted for testing
- purposes.\\
- Arch Linux \citep{website:archlinux} was installed and configured to run
- the medium scale setup. Unfortunately the proprietary Dante
- \citep{website:audinate} driver for Linux, offered by Four Audio
- \citep{website:fouraudio}, creates non-trivial circumstances for using it
- on an up-to-date Linux kernel, due to \gls{alsa} \gls{api} changes not
- accounted for.\\
- While the current \gls{os} - an Ubuntu \citep{website:ubuntu} Studio
- 2012 \gls{lts} - still runs well in its own parameters, its support has run
- out and it is therefore becoming harder, if not impossible, to build
- newer software on it, using newer versions of free software compilers.\\
- For research purposes however, it is desirable to be able to try new
- kernel and software features, finding the most stable and secure setup
- possible involving realtime enabled kernels and building new versions of
- (spatialisation) software on a regular basis.\\
- The hardware of the large scale setup at TU Berlin in room H0104 was
- being updated and unusable at the time of writing. In the future it will
- however become a valuable candidate for testing of the sought after
- \gls{ssr} features, as its setup involves no Dante network, but is
- instead run by several rendering computers connected to \gls{madi} and
- \gls{adat} lightpipe enabled speaker systems.\\
- Although a \gls{wfs} setup for testing purposes was eligible, it
- generally is not needed for implementing (a subset of) the features
- described in \nameref{sec:implementation}, as they can be tested using
- two machines running Linux, \gls{jack} and a development version of
- the \gls{ssr}.\\
+ This section covers the implementation of a networking interface for the
+ \gls{ssr} and the considerations leading to it.
+ The application was chosen for extension, as it offers a wide set of
+ rendering algorithms (in various stages of completion) by using the
+ \gls{apf} \citep{MatthiasGeierTorbenHohn1890} and because it was still
+ actively maintained by its creators at the time of writing.\\
\subsection{Outline}
\label{subsec:outline}
- As described in \nameref{sec:implementation}, initially extending the
- \gls{ssr}'s features was aimed at in the scope of creating a replacement
- for the aging sWONDER software, enabling it to run networked instances to
- drive a medium or large scale \gls{wfs} setup.\\
+ Initially it was aimed at extending the \gls{ssr}'s features in the scope
+ of creating a replacement for the aging sWONDER software, enabling it to
+ run networked instances to drive a medium or large scale \gls{wfs} setup.
+ The approach appeared too narrow however, as the application offers many
+ different rendering algorithms. A networking extension therefore would
+ have to be available to all of them with an equal feature set.
+ Additionally, extending a rendering framework by a networking feature,
+ with the help of only one of its engines proved to be linked to a
+ massive, but avoidable overhead (see~\ref{subsubsec:preliminaries}).\\
+ The \gls{ssr}, being a multi-purpose spatial audio renderer, can be used
+ in diverse setup scenarios (see~\ref{subsubsec:setups}), which is why not
+ only classic server-client relationships
+ (see~\ref{subsubsec:remote_controlling_a_server}), but also client-only
+ and local (see~\ref{subsubsec:remote_controlling_a_client}) setups had to
+ be taken account of. In addition the case of medium and
+ large scale loudspeaker based rendering setups and their specifics had to
+ be considered (see~\ref{subsubsec:rendering_on_dedicated_speakers}).
+
+ \subsubsection{Prelimenaries}
+ \label{subsubsec:preliminaries}
+ In preparation to this work, an implemention of a side-by-side
+ installation to the \gls{os} currently driving the \gls{wfs} system
+ setup of the Electronic Studio at TU Berlin
+ \citep{website:tu-electronic_studio} was attempted for testing
+ purposes.\\
+ Arch Linux \citep{website:archlinux} was installed and configured to
+ run the medium scale setup. Unfortunately the proprietary Dante
+ \citep{website:audinate} driver for Linux, offered by Four Audio
+ \citep{website:fouraudio}, creates non-trivial circumstances for using
+ it on an up-to-date Linux kernel, due to \gls{alsa} \gls{api} changes
+ not accounted for.\\
+ While the current \gls{os} - an Ubuntu \citep{website:ubuntu} Studio
+ 2012 \gls{lts} - still runs well in its own parameters, its support has
+ run out and it is therefore becoming harder, if not impossible, to
+ build newer software on it, using newer versions of free software
+ compilers.\\
+ For research purposes however, it is desirable to be able to try new
+ kernel and software features, finding the most stable and secure setup
+ possible involving realtime enabled kernels and building new versions
+ of (spatialisation) software on a regular basis.\\
+ The hardware of the large scale setup at TU Berlin in room H0104 was
+ being updated and unusable at the time of writing. In the future it
+ will however become a valuable candidate for testing of the sought
+ after \gls{ssr} features, as its setup involves no Dante network, but
+ is instead run by several rendering computers connected to \gls{madi}
+ and \gls{adat} lightpipe enabled speaker systems.\\
+ Although a \gls{wfs} setup for testing purposes was eligible, it
+ generally is not needed for implementing the features described in
+ the following sections and subsections, as they can be tested using two
+ machines running Linux, \gls{jack} and a development version of the
+ \gls{ssr}.\\
\subsubsection{Remote controlling a server}
\label{subsubsec:remote_controlling_a_server}
@@ -497,6 +485,11 @@ parskip=never]{paper}
before sending a message to the \gls{ssr}. Additionally the message has
to be linted (error checked) before sending and again parsed, after
receiving an answer from the application.\\
+ The \gls{ip} interface achieves to offer more or less direct access to
+ the \nameref{subsec:publisher_subscriber_interface}. It has however, no
+ notion of a networked setup and could therefore be described as a
+ two-directional message system between two destinations. With it, only
+ setups with up to \textit{n} clients are possible.
\subsubsection{OSC through PureData}
\label{subsubsec:osc_through_puredata}
@@ -516,7 +509,6 @@ parskip=never]{paper}
its subscribers, too.\\
\cleardoublepage
-
\subsection{Open Sound Control interface}
\label{subsec:osc-interface}
The networking interface conceived in the course of this work was
@@ -544,7 +536,11 @@ parskip=never]{paper}
A nearly configuration-less approach, based on subscribing clients on
sending poll messages to them proved more open (in the sense that it can
be interfaced with by any \gls{osc}-capable application or programming
- language) and have less configuration overhead.\\
+ language) and have less configuration overhead. With it, a diverse set of
+ setups can be achieved (further described in~\ref{subsubsec:setups}),
+ which at the same time remain dynamically configurable (using a plethora
+ of \gls{osc} implementations) and debuggable using tests (further
+ explored in~\ref{subsec:automated_tests}).
\begin{figure}[!htb]
\centering
@@ -768,11 +764,24 @@ ssr-aap -N “server” -C “127.0.0.1:50002”
Additionally it's possible for clients (\gls{ssr} client instances,
or \gls{osc} capable applications) to subscribe to the server
instance, or be subscribed to it by another client, using a message
- level system further explained in \nameref{subsubsec:message_levels}.
+ level system further explained in~\ref{subsubsec:message_levels}.
Every valid \gls{osc} message sent to the server instance will be
delegated to all of its clients upon evaluation, again according to
the aforementioned message level system.
+ \paragraph{Verbosity}
+ \label{para:verbosity}
+ The \gls{ssr} can be started with several levels of verbosity. These
+ are accessed by using the flag \textbf{-v}, up to three times (i.e.
+ \textbf{-vvv}).\\
+ The higher the level of verbosity, the more messages will be printed
+ by the application. This especially applies to the \gls{osc}
+ interface part of the \gls{ssr}, as most incoming and outgoing
+ messages will be printed to \gls{stdout} at a level of \textbf{-vv}.
+ At a level of \textbf{-vvv}, additionally all incoming and outgoing
+ messages, that are issued in very short intervals per default
+ (see~\ref{subsubsec:message_levels} for details) will be printed.
+
\subsubsection{Setups}
\label{subsubsec:setups}
The \gls{ssr} offers the possibility for many different \gls{osc}
@@ -781,7 +790,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002”
All examples provide audio input via a \gls{jack} client, which can be
local (on each client's or server's host computer) or provided through
external audio inputs from another host computer (e.g.\ through
- \gls{adat} orj \gls{madi}). This however is not mandatory, as the
+ \gls{adat} or \gls{madi}). This however is not mandatory, as the
\gls{ssr} is capable of playing back audio files directly.\\
The differences between server and client messaging is further
elaborated upon in \textbf{\nameref{subsubsec:message_interface}}.\\
@@ -1393,13 +1402,13 @@ ssr-aap -N “server” -C “127.0.0.1:50002”
\subsection{Automated tests}
\label{subsec:automated_tests}
- To probe the \gls{osc} interface's robustness automatically, a set of
- tests were written in \gls{sclang}. The \gls{ssr} was developed without
- the help of a test framework, which is responsible to test its
- components, after they have been changed (usually before compiling the
- source to binary). This means, that internal (e.g.\ the \gls{pubsub}
- interface), or external (e.g.\ the \gls{ip} or \gls{osc} interface)
- functionality might or might not work as expected.\\
+ The \gls{ssr} was developed without the help of a test framework, which
+ is responsible to test its components, after they have been changed.
+ This means, that internal (e.g.\ the \gls{pubsub} interface), or external
+ (e.g.\ the \gls{ip} or \gls{osc} interface) functionality might or might
+ not work as expected. To test the \gls{osc} interface's logical coherency
+ and robustness automatically, a set of tests were written in
+ \gls{sclang}.\\
The tests are divided into those probing robustness of the \gls{osc}
interface and others probing its functionality. The robustness tests
further divide into server and client specific tests, where authorized
@@ -1407,100 +1416,128 @@ ssr-aap -N “server” -C “127.0.0.1:50002”
tests for general operability, i.e.\ testing certain features or
workflows once and long-running tests, where features are tried
repeatedly.\\
- Listing~\ref{lst:ssr-tests-sclang-unsubscribed-controls-server}
- and~\ref{lst:ssr-tests-sclang-subscribed-controls-server} describe
- server-side tests for robustness. While the first will not lead to any
- processed action by the server, the latter will. This is explained by
- \gls{sclang} not being a subscribed client with a \textit{MessageLevel}
- of \textit{SERVER} or higher in the first case. In the second test
- however \gls{sclang} subscribes to the \gls{ssr} server instance, which
- is why the \gls{osc} messages are evaluated in this case.\\
-
- \begin{listing}[!htb]
- \begin{mdframed}
- \inputminted[numbers=left, firstline=145, lastline=158,
- fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd}
- \end{mdframed}
- \caption{supercollider/tests.scd: \gls{sclang} (unsubscribed) tries to
- control a \gls{ssr} server}
- \label{lst:ssr-tests-sclang-unsubscribed-controls-server}
- \end{listing}\\
-
- \begin{listing}[!htb]
- \begin{mdframed}
- \inputminted[numbers=left, firstline=160, lastline=183,
- fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd}
- \end{mdframed}
- \caption{supercollider/tests.scd: \gls{sclang} (subscribed) tries to
- control a \gls{ssr} server}
- \label{lst:ssr-tests-sclang-subscribed-controls-server}
- \end{listing}\\
-
- The tests described in
- Listing~\ref{lst:ssr-tests-sclang-controls-client-unpolled}
- and~\ref{lst:ssr-tests-sclang-controls-client-polled} are client-side
- tests for robustness, that work in a similar fashion to the
- aforementioned server-side tests. While in the first case, the sent
- \gls{osc} messages are not evaluated, because \gls{sclang}, mimicking a
- server instance (see~\ref{para:server_mimicry}), did not poll the
- \gls{ssr} client instance up front, in the second case the messages are
- evaluated, because it did poll the client first.
-
- \begin{listing}[!htb]
- \begin{mdframed}
- \inputminted[numbers=left, firstline=185, lastline=197,
- fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd}
- \end{mdframed}
- \caption{supercollider/tests.scd: \gls{sclang} tries to control a
- \gls{ssr} client (without polling it)}
- \label{lst:ssr-tests-sclang-controls-client-unpolled}
- \end{listing}\\
-
- \begin{listing}[!htb]
- \begin{mdframed}
- \inputminted[numbers=left, firstline=199, lastline=214,
- fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd}
- \end{mdframed}
- \caption{supercollider/tests.scd: \gls{sclang} tries to control a
- \gls{ssr} client (with previously polling it)}
- \label{lst:ssr-tests-sclang-controls-client-polled}
- \end{listing}\\
-
- In all tests for robustness, the attempt is made to break the
- implementation of the \nameref{subsubsec:message_interface}. This is
- achieved by purposely using ranges of data types for messages, that are
- not allowed, or not defined in the \gls{ssr}'s internal implementation.\\
- Two examples for weak spot exploitations are the use of negative integers
- for \glspl{id} in source related messages (only non-zero, non-negative
- \glspl{id} are allowed internally) or supplying an empty string as
- hostname or port number for subscription messages.\\
- The first example will lead to undefined behavior, if the range is not
- checked in the implementation, as a \textit{static\_cast} is used
- internally to cast the value of the message data type (\textit{unsigned
- int}) to the one expected by the \gls{ssr}'s Controller implementation
- (\textit{signed int}) and the outcome of said operation is implementation
- dependant (depending on the \gls{os} in use).\\
- The second example, if not checked for empty string, would lead to the
- \gls{osc} interface trying to send poll messages out to a possibly
- defective address.\\
-
- \begin{listing}[!htb]
- \begin{mdframed}
- \inputminted[numbers=left, firstline=216, lastline=225,
- fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd}
- \end{mdframed}
- \caption{supercollider/tests.scd: \gls{sclang} controls a \gls{ssr}
- client (with previously polling it), creating several sources and
- moving them}
- \label{lst:ssr-tests-sclang-sources}
- \end{listing}\\
-
- The test described in Listing~\ref{lst:ssr-tests-sclang-sources} is a
- test for functionality, which also serves as a long-running stress test
- for the \gls{ssr}. It creates 20 sources, that are then moved around
- randomly for 100 seconds, every 100ms, which on a Lenovo W540, with an
- Intel i7-4700MQ and 16Gb RAM creates less than 50\% of \gls{cpu} load.\\
+ \subsubsection{Robustness}
+ \label{subsubsec:robustness}
+
+ Listing~\ref{lst:ssr-tests-sclang-unsubscribed-controls-server}
+ and~\ref{lst:ssr-tests-sclang-subscribed-controls-server} describe
+ server-side tests for robustness. While the first will not lead to any
+ processed action by the server, the latter will. This is explained by
+ \gls{sclang} not being a subscribed client with a \textit{MessageLevel}
+ of \textit{SERVER} or higher in the first case. In the second test
+ however \gls{sclang} subscribes to the \gls{ssr} server instance, which
+ is why the \gls{osc} messages are evaluated in this case.\\
+
+ \begin{listing}[!htb]
+ \begin{mdframed}
+ \inputminted[numbers=left, firstline=145, lastline=158,
+ fontsize=\footnotesize]{supercollider}
+ {../../ssr/supercollider/tests.scd}
+ \end{mdframed}
+ \caption{supercollider/tests.scd: \gls{sclang} (unsubscribed) tries
+ to control a \gls{ssr} server}
+ \label{lst:ssr-tests-sclang-unsubscribed-controls-server}
+ \end{listing}\\
+
+ \begin{listing}[!htb]
+ \begin{mdframed}
+ \inputminted[numbers=left, firstline=160, lastline=183,
+ fontsize=\footnotesize]{supercollider}
+ {../../ssr/supercollider/tests.scd}
+ \end{mdframed}
+ \caption{supercollider/tests.scd: \gls{sclang} (subscribed) tries to
+ control a \gls{ssr} server}
+ \label{lst:ssr-tests-sclang-subscribed-controls-server}
+ \end{listing}\\
+
+ The tests described in
+ Listing~\ref{lst:ssr-tests-sclang-controls-client-unpolled}
+ and~\ref{lst:ssr-tests-sclang-controls-client-polled} are client-side
+ tests for robustness, that work in a similar fashion to the
+ aforementioned server-side tests. While in the first case, the sent
+ \gls{osc} messages are not evaluated, because \gls{sclang}, mimicking a
+ server instance (see~\ref{para:server_mimicry}), did not poll the
+ \gls{ssr} client instance up front, in the second case the messages are
+ evaluated, because it did poll the client first.
+
+ \begin{listing}[!htb]
+ \begin{mdframed}
+ \inputminted[numbers=left, firstline=185, lastline=197,
+ fontsize=\footnotesize]{supercollider}
+ {../../ssr/supercollider/tests.scd}
+ \end{mdframed}
+ \caption{supercollider/tests.scd: \gls{sclang} tries to control a
+ \gls{ssr} client (without polling it)}
+ \label{lst:ssr-tests-sclang-controls-client-unpolled}
+ \end{listing}\\
+
+ \begin{listing}[!htb]
+ \begin{mdframed}
+ \inputminted[numbers=left, firstline=199, lastline=214,
+ fontsize=\footnotesize]{supercollider}
+ {../../ssr/supercollider/tests.scd}
+ \end{mdframed}
+ \caption{supercollider/tests.scd: \gls{sclang} tries to control a
+ \gls{ssr} client (with previously polling it)}
+ \label{lst:ssr-tests-sclang-controls-client-polled}
+ \end{listing}\\
+
+ In all tests for robustness, the attempt is made to break the
+ implementation of the \nameref{subsubsec:message_interface}. This is
+ achieved by purposely using ranges of data types for messages, that are
+ not allowed, or not defined in the \gls{ssr}'s internal
+ implementation.\\
+ Two examples for weak spot exploitations are the use of negative
+ integers for \glspl{id} in source related messages (only non-zero,
+ non-negative \glspl{id} are allowed internally) or supplying an empty
+ string as hostname or port number for subscription messages.\\
+ The first example will lead to undefined behavior, if the range is not
+ checked in the implementation, as a \textit{static\_cast} is used
+ internally to cast the value of the message data type (\textit{unsigned
+ int}) to the one expected by the \gls{ssr}'s Controller implementation
+ (\textit{signed int}) and the outcome of said operation is
+ implementation dependant (depending on the \gls{os} in use).\\
+ The second example, if not checked for empty string, would lead to the
+ \gls{osc} interface trying to create a possibly defective address and
+ send poll messages out to it.\\
+ While only some of the above mentioned scenarios could lead to a crash
+ of the program (under certain circumstances), unhandled all of them
+ waste ressources, which is undesired.
+ To circumvent possibly harmful input using the \gls{osc} interface, a
+ set of sanity checks were implemented, that only allow for a received
+ message to be processed, if all of its components fit the requirements.
+
+ \subsubsection{Functionality and operability}
+ \label{subsubsec:functionality_and_operability}
+
+ \begin{listing}[!htb]
+ \begin{mdframed}
+ \inputminted[numbers=left, firstline=216, lastline=225,
+ fontsize=\footnotesize]{supercollider}
+ {../../ssr/supercollider/tests.scd}
+ \end{mdframed}
+ \caption{supercollider/tests.scd: \gls{sclang} controls a \gls{ssr}
+ client (with previously polling it), creating several sources and
+ moving them}
+ \label{lst:ssr-tests-sclang-sources}
+ \end{listing}\\
+ The test described in Listing~\ref{lst:ssr-tests-sclang-sources} is a
+ test for functionality, which also serves as a long-running stress test
+ for the \gls{ssr}. It creates 20 sources, that are then moved around
+ randomly, every 100ms, for 100 seconds, which on a Lenovo W540, with an
+ Intel i7-4700MQ and 16Gb RAM created less than 50\% of \gls{cpu}
+ load.\\
+ Based on the above mentioned tests, the basic functionality of the
+ \gls{osc} interface can be guaranteed and depending on the host's
+ hardware also a maximum degree of capacity utilization can be
+ estimated, when observing the \gls{ssr}'s workload towards the system,
+ while using the long-running tests.\\
+ It has to be mentioned, that a higher load can be observed, when using
+ higher levels of verbosity (especially above \textit{-vv}). This is
+ explained by the fact, that the \gls{ssr} will print out every
+ \gls{osc} message received and sent above the aforementioned verbosity
+ level.
\cleardoublepage
\section{Discussion}