diff options
Diffstat (limited to 'thesis')
| -rw-r--r-- | thesis/thesis.tex | 401 |
1 files changed, 219 insertions, 182 deletions
diff --git a/thesis/thesis.tex b/thesis/thesis.tex index 2b6c4ff..6a163e8 100644 --- a/thesis/thesis.tex +++ b/thesis/thesis.tex @@ -44,6 +44,7 @@ parskip=never]{paper} \newacronym{aap}{AAP}{Ambisonics Amplitude Panning} \newacronym{adat}{ADAT}{Alesis Digital Audio Tape} \newacronym{alsa}{ALSA}{Advanced Linux Sound Architecture} +\newacronym{apf}{APF}{Audio Processing Framework} \newacronym{api}{API}{Application programming interface} \newacronym{asdf}{ASDF}{Audio Scene Description Format} \newacronym{bs}{BS}{Binaural Synthesis} @@ -78,13 +79,19 @@ parskip=never]{paper} \newglossaryentry{sclang}{ name={sclang}, description={Name of the SuperCollider programming language and the - interpreter executable of the SuperCollider programming language.} + interpreter executable of the SuperCollider programming language} } \newglossaryentry{id}{ name={ID}, - description={A name or number, that identifies an object.}, + description={A name or number, that identifies an object}, plural=IDs } +\newglossaryentry{stdout}{ + name={stdout}, + description={The standard output is a stream where a program writes its + output data to. This can be a log file or a terminal}, +} + \makeglossaries @@ -346,85 +353,66 @@ parskip=never]{paper} \cleardoublepage \section{Implementation} \label{sec:implementation} - The \gls{ssr}, due to its diverse set of rendering engines, which are made - available through an extensible framework, and its relatively clean - codebase, is a good candidate for future large scale \gls{wfs} setups. These type - of features are not yet implemented though and will need testing.\\ - Therefore I propose the implementation and testing of said feature, making - the \gls{ssr} capable of rendering on large scale \gls{wfs} setups with many nodes, - controlled by a master instance.\\ - The sought implementation is inspired by the architecture of sWONDER, but - instead of creating many single purpose applications, the master/node - feature will be made available through flags to the \gls{ssr} executable, when - starting it. This behavior is already actively harnessed eg.\ for selecting - one of the several rendering engines. - \begin{figure}[!htb] - \centering - \includegraphics[scale=0.9, trim = 31mm 190mm 24mm 8mm, clip] - {ssr-networking.pdf} - \caption{A diagram displaying the \gls{ssr} master/node setup with TCP/IP - socket connections over network (green lines), audio channels (red - dashed lines) and \gls{osc} connection (blue dashed line). Machines are - indicated as red dashed rectangles and connections to audio hardware - as outputs of \gls{ssr} nodes as black lines below them.} - \label{fig:ssr-networking} - \end{figure} - While the \gls{ssr} already has an internal logic to know which loudspeaker will - be used for what virtual audio source, this will have to be extended to be - able to know which renderer node has to render what source on which - loudspeaker (see Figure~\ref{fig:ssr-networking}). - To achieve the above features, the \gls{ssr}'s messaging (and thus also settings) - capabilities have to be extended alongside its internal logic concerning - the selection of output channels (and the master to node notification - thereof). To introduce as little redundant code as possible, most likely a - ``the client knows all'' setup is desirable, in which each node knows about - the whole setup, but is also set to only serve its own subset of - loudspeakers in it. This will make sure that the rendering engine remains - functional also in a small scale \gls{wfs} setup.\\ - The lack of a direct \gls{osc} functionality, as provided by the two other - renderers, will not be problematic, as master and nodes can communicate - through their builtin TCP/IP sockets directly and the master can, if - needed, be controlled via \gls{osc}. - - \subsection{Prelimenaries} - \label{subsec:preliminaries} - In preparation to this work, an implemention of a side-by-side - installation to the \gls{os} currently driving the \gls{wfs} system setup - of the Electronic Studio at TU Berlin - \citep{website:tu-electronic_studio} was attempted for testing - purposes.\\ - Arch Linux \citep{website:archlinux} was installed and configured to run - the medium scale setup. Unfortunately the proprietary Dante - \citep{website:audinate} driver for Linux, offered by Four Audio - \citep{website:fouraudio}, creates non-trivial circumstances for using it - on an up-to-date Linux kernel, due to \gls{alsa} \gls{api} changes not - accounted for.\\ - While the current \gls{os} - an Ubuntu \citep{website:ubuntu} Studio - 2012 \gls{lts} - still runs well in its own parameters, its support has run - out and it is therefore becoming harder, if not impossible, to build - newer software on it, using newer versions of free software compilers.\\ - For research purposes however, it is desirable to be able to try new - kernel and software features, finding the most stable and secure setup - possible involving realtime enabled kernels and building new versions of - (spatialisation) software on a regular basis.\\ - The hardware of the large scale setup at TU Berlin in room H0104 was - being updated and unusable at the time of writing. In the future it will - however become a valuable candidate for testing of the sought after - \gls{ssr} features, as its setup involves no Dante network, but is - instead run by several rendering computers connected to \gls{madi} and - \gls{adat} lightpipe enabled speaker systems.\\ - Although a \gls{wfs} setup for testing purposes was eligible, it - generally is not needed for implementing (a subset of) the features - described in \nameref{sec:implementation}, as they can be tested using - two machines running Linux, \gls{jack} and a development version of - the \gls{ssr}.\\ + This section covers the implementation of a networking interface for the + \gls{ssr} and the considerations leading to it. + The application was chosen for extension, as it offers a wide set of + rendering algorithms (in various stages of completion) by using the + \gls{apf} \citep{MatthiasGeierTorbenHohn1890} and because it was still + actively maintained by its creators at the time of writing.\\ \subsection{Outline} \label{subsec:outline} - As described in \nameref{sec:implementation}, initially extending the - \gls{ssr}'s features was aimed at in the scope of creating a replacement - for the aging sWONDER software, enabling it to run networked instances to - drive a medium or large scale \gls{wfs} setup.\\ + Initially it was aimed at extending the \gls{ssr}'s features in the scope + of creating a replacement for the aging sWONDER software, enabling it to + run networked instances to drive a medium or large scale \gls{wfs} setup. + The approach appeared too narrow however, as the application offers many + different rendering algorithms. A networking extension therefore would + have to be available to all of them with an equal feature set. + Additionally, extending a rendering framework by a networking feature, + with the help of only one of its engines proved to be linked to a + massive, but avoidable overhead (see~\ref{subsubsec:preliminaries}).\\ + The \gls{ssr}, being a multi-purpose spatial audio renderer, can be used + in diverse setup scenarios (see~\ref{subsubsec:setups}), which is why not + only classic server-client relationships + (see~\ref{subsubsec:remote_controlling_a_server}), but also client-only + and local (see~\ref{subsubsec:remote_controlling_a_client}) setups had to + be taken account of. In addition the case of medium and + large scale loudspeaker based rendering setups and their specifics had to + be considered (see~\ref{subsubsec:rendering_on_dedicated_speakers}). + + \subsubsection{Prelimenaries} + \label{subsubsec:preliminaries} + In preparation to this work, an implemention of a side-by-side + installation to the \gls{os} currently driving the \gls{wfs} system + setup of the Electronic Studio at TU Berlin + \citep{website:tu-electronic_studio} was attempted for testing + purposes.\\ + Arch Linux \citep{website:archlinux} was installed and configured to + run the medium scale setup. Unfortunately the proprietary Dante + \citep{website:audinate} driver for Linux, offered by Four Audio + \citep{website:fouraudio}, creates non-trivial circumstances for using + it on an up-to-date Linux kernel, due to \gls{alsa} \gls{api} changes + not accounted for.\\ + While the current \gls{os} - an Ubuntu \citep{website:ubuntu} Studio + 2012 \gls{lts} - still runs well in its own parameters, its support has + run out and it is therefore becoming harder, if not impossible, to + build newer software on it, using newer versions of free software + compilers.\\ + For research purposes however, it is desirable to be able to try new + kernel and software features, finding the most stable and secure setup + possible involving realtime enabled kernels and building new versions + of (spatialisation) software on a regular basis.\\ + The hardware of the large scale setup at TU Berlin in room H0104 was + being updated and unusable at the time of writing. In the future it + will however become a valuable candidate for testing of the sought + after \gls{ssr} features, as its setup involves no Dante network, but + is instead run by several rendering computers connected to \gls{madi} + and \gls{adat} lightpipe enabled speaker systems.\\ + Although a \gls{wfs} setup for testing purposes was eligible, it + generally is not needed for implementing the features described in + the following sections and subsections, as they can be tested using two + machines running Linux, \gls{jack} and a development version of the + \gls{ssr}.\\ \subsubsection{Remote controlling a server} \label{subsubsec:remote_controlling_a_server} @@ -497,6 +485,11 @@ parskip=never]{paper} before sending a message to the \gls{ssr}. Additionally the message has to be linted (error checked) before sending and again parsed, after receiving an answer from the application.\\ + The \gls{ip} interface achieves to offer more or less direct access to + the \nameref{subsec:publisher_subscriber_interface}. It has however, no + notion of a networked setup and could therefore be described as a + two-directional message system between two destinations. With it, only + setups with up to \textit{n} clients are possible. \subsubsection{OSC through PureData} \label{subsubsec:osc_through_puredata} @@ -516,7 +509,6 @@ parskip=never]{paper} its subscribers, too.\\ \cleardoublepage - \subsection{Open Sound Control interface} \label{subsec:osc-interface} The networking interface conceived in the course of this work was @@ -544,7 +536,11 @@ parskip=never]{paper} A nearly configuration-less approach, based on subscribing clients on sending poll messages to them proved more open (in the sense that it can be interfaced with by any \gls{osc}-capable application or programming - language) and have less configuration overhead.\\ + language) and have less configuration overhead. With it, a diverse set of + setups can be achieved (further described in~\ref{subsubsec:setups}), + which at the same time remain dynamically configurable (using a plethora + of \gls{osc} implementations) and debuggable using tests (further + explored in~\ref{subsec:automated_tests}). \begin{figure}[!htb] \centering @@ -768,11 +764,24 @@ ssr-aap -N “server” -C “127.0.0.1:50002” Additionally it's possible for clients (\gls{ssr} client instances, or \gls{osc} capable applications) to subscribe to the server instance, or be subscribed to it by another client, using a message - level system further explained in \nameref{subsubsec:message_levels}. + level system further explained in~\ref{subsubsec:message_levels}. Every valid \gls{osc} message sent to the server instance will be delegated to all of its clients upon evaluation, again according to the aforementioned message level system. + \paragraph{Verbosity} + \label{para:verbosity} + The \gls{ssr} can be started with several levels of verbosity. These + are accessed by using the flag \textbf{-v}, up to three times (i.e. + \textbf{-vvv}).\\ + The higher the level of verbosity, the more messages will be printed + by the application. This especially applies to the \gls{osc} + interface part of the \gls{ssr}, as most incoming and outgoing + messages will be printed to \gls{stdout} at a level of \textbf{-vv}. + At a level of \textbf{-vvv}, additionally all incoming and outgoing + messages, that are issued in very short intervals per default + (see~\ref{subsubsec:message_levels} for details) will be printed. + \subsubsection{Setups} \label{subsubsec:setups} The \gls{ssr} offers the possibility for many different \gls{osc} @@ -781,7 +790,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” All examples provide audio input via a \gls{jack} client, which can be local (on each client's or server's host computer) or provided through external audio inputs from another host computer (e.g.\ through - \gls{adat} orj \gls{madi}). This however is not mandatory, as the + \gls{adat} or \gls{madi}). This however is not mandatory, as the \gls{ssr} is capable of playing back audio files directly.\\ The differences between server and client messaging is further elaborated upon in \textbf{\nameref{subsubsec:message_interface}}.\\ @@ -1393,13 +1402,13 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \subsection{Automated tests} \label{subsec:automated_tests} - To probe the \gls{osc} interface's robustness automatically, a set of - tests were written in \gls{sclang}. The \gls{ssr} was developed without - the help of a test framework, which is responsible to test its - components, after they have been changed (usually before compiling the - source to binary). This means, that internal (e.g.\ the \gls{pubsub} - interface), or external (e.g.\ the \gls{ip} or \gls{osc} interface) - functionality might or might not work as expected.\\ + The \gls{ssr} was developed without the help of a test framework, which + is responsible to test its components, after they have been changed. + This means, that internal (e.g.\ the \gls{pubsub} interface), or external + (e.g.\ the \gls{ip} or \gls{osc} interface) functionality might or might + not work as expected. To test the \gls{osc} interface's logical coherency + and robustness automatically, a set of tests were written in + \gls{sclang}.\\ The tests are divided into those probing robustness of the \gls{osc} interface and others probing its functionality. The robustness tests further divide into server and client specific tests, where authorized @@ -1407,100 +1416,128 @@ ssr-aap -N “server” -C “127.0.0.1:50002” tests for general operability, i.e.\ testing certain features or workflows once and long-running tests, where features are tried repeatedly.\\ - Listing~\ref{lst:ssr-tests-sclang-unsubscribed-controls-server} - and~\ref{lst:ssr-tests-sclang-subscribed-controls-server} describe - server-side tests for robustness. While the first will not lead to any - processed action by the server, the latter will. This is explained by - \gls{sclang} not being a subscribed client with a \textit{MessageLevel} - of \textit{SERVER} or higher in the first case. In the second test - however \gls{sclang} subscribes to the \gls{ssr} server instance, which - is why the \gls{osc} messages are evaluated in this case.\\ - - \begin{listing}[!htb] - \begin{mdframed} - \inputminted[numbers=left, firstline=145, lastline=158, - fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd} - \end{mdframed} - \caption{supercollider/tests.scd: \gls{sclang} (unsubscribed) tries to - control a \gls{ssr} server} - \label{lst:ssr-tests-sclang-unsubscribed-controls-server} - \end{listing}\\ - - \begin{listing}[!htb] - \begin{mdframed} - \inputminted[numbers=left, firstline=160, lastline=183, - fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd} - \end{mdframed} - \caption{supercollider/tests.scd: \gls{sclang} (subscribed) tries to - control a \gls{ssr} server} - \label{lst:ssr-tests-sclang-subscribed-controls-server} - \end{listing}\\ - - The tests described in - Listing~\ref{lst:ssr-tests-sclang-controls-client-unpolled} - and~\ref{lst:ssr-tests-sclang-controls-client-polled} are client-side - tests for robustness, that work in a similar fashion to the - aforementioned server-side tests. While in the first case, the sent - \gls{osc} messages are not evaluated, because \gls{sclang}, mimicking a - server instance (see~\ref{para:server_mimicry}), did not poll the - \gls{ssr} client instance up front, in the second case the messages are - evaluated, because it did poll the client first. - - \begin{listing}[!htb] - \begin{mdframed} - \inputminted[numbers=left, firstline=185, lastline=197, - fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd} - \end{mdframed} - \caption{supercollider/tests.scd: \gls{sclang} tries to control a - \gls{ssr} client (without polling it)} - \label{lst:ssr-tests-sclang-controls-client-unpolled} - \end{listing}\\ - - \begin{listing}[!htb] - \begin{mdframed} - \inputminted[numbers=left, firstline=199, lastline=214, - fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd} - \end{mdframed} - \caption{supercollider/tests.scd: \gls{sclang} tries to control a - \gls{ssr} client (with previously polling it)} - \label{lst:ssr-tests-sclang-controls-client-polled} - \end{listing}\\ - - In all tests for robustness, the attempt is made to break the - implementation of the \nameref{subsubsec:message_interface}. This is - achieved by purposely using ranges of data types for messages, that are - not allowed, or not defined in the \gls{ssr}'s internal implementation.\\ - Two examples for weak spot exploitations are the use of negative integers - for \glspl{id} in source related messages (only non-zero, non-negative - \glspl{id} are allowed internally) or supplying an empty string as - hostname or port number for subscription messages.\\ - The first example will lead to undefined behavior, if the range is not - checked in the implementation, as a \textit{static\_cast} is used - internally to cast the value of the message data type (\textit{unsigned - int}) to the one expected by the \gls{ssr}'s Controller implementation - (\textit{signed int}) and the outcome of said operation is implementation - dependant (depending on the \gls{os} in use).\\ - The second example, if not checked for empty string, would lead to the - \gls{osc} interface trying to send poll messages out to a possibly - defective address.\\ - - \begin{listing}[!htb] - \begin{mdframed} - \inputminted[numbers=left, firstline=216, lastline=225, - fontsize=\footnotesize]{supercollider}{../../ssr/supercollider/tests.scd} - \end{mdframed} - \caption{supercollider/tests.scd: \gls{sclang} controls a \gls{ssr} - client (with previously polling it), creating several sources and - moving them} - \label{lst:ssr-tests-sclang-sources} - \end{listing}\\ - - The test described in Listing~\ref{lst:ssr-tests-sclang-sources} is a - test for functionality, which also serves as a long-running stress test - for the \gls{ssr}. It creates 20 sources, that are then moved around - randomly for 100 seconds, every 100ms, which on a Lenovo W540, with an - Intel i7-4700MQ and 16Gb RAM creates less than 50\% of \gls{cpu} load.\\ + \subsubsection{Robustness} + \label{subsubsec:robustness} + + Listing~\ref{lst:ssr-tests-sclang-unsubscribed-controls-server} + and~\ref{lst:ssr-tests-sclang-subscribed-controls-server} describe + server-side tests for robustness. While the first will not lead to any + processed action by the server, the latter will. This is explained by + \gls{sclang} not being a subscribed client with a \textit{MessageLevel} + of \textit{SERVER} or higher in the first case. In the second test + however \gls{sclang} subscribes to the \gls{ssr} server instance, which + is why the \gls{osc} messages are evaluated in this case.\\ + + \begin{listing}[!htb] + \begin{mdframed} + \inputminted[numbers=left, firstline=145, lastline=158, + fontsize=\footnotesize]{supercollider} + {../../ssr/supercollider/tests.scd} + \end{mdframed} + \caption{supercollider/tests.scd: \gls{sclang} (unsubscribed) tries + to control a \gls{ssr} server} + \label{lst:ssr-tests-sclang-unsubscribed-controls-server} + \end{listing}\\ + + \begin{listing}[!htb] + \begin{mdframed} + \inputminted[numbers=left, firstline=160, lastline=183, + fontsize=\footnotesize]{supercollider} + {../../ssr/supercollider/tests.scd} + \end{mdframed} + \caption{supercollider/tests.scd: \gls{sclang} (subscribed) tries to + control a \gls{ssr} server} + \label{lst:ssr-tests-sclang-subscribed-controls-server} + \end{listing}\\ + + The tests described in + Listing~\ref{lst:ssr-tests-sclang-controls-client-unpolled} + and~\ref{lst:ssr-tests-sclang-controls-client-polled} are client-side + tests for robustness, that work in a similar fashion to the + aforementioned server-side tests. While in the first case, the sent + \gls{osc} messages are not evaluated, because \gls{sclang}, mimicking a + server instance (see~\ref{para:server_mimicry}), did not poll the + \gls{ssr} client instance up front, in the second case the messages are + evaluated, because it did poll the client first. + + \begin{listing}[!htb] + \begin{mdframed} + \inputminted[numbers=left, firstline=185, lastline=197, + fontsize=\footnotesize]{supercollider} + {../../ssr/supercollider/tests.scd} + \end{mdframed} + \caption{supercollider/tests.scd: \gls{sclang} tries to control a + \gls{ssr} client (without polling it)} + \label{lst:ssr-tests-sclang-controls-client-unpolled} + \end{listing}\\ + + \begin{listing}[!htb] + \begin{mdframed} + \inputminted[numbers=left, firstline=199, lastline=214, + fontsize=\footnotesize]{supercollider} + {../../ssr/supercollider/tests.scd} + \end{mdframed} + \caption{supercollider/tests.scd: \gls{sclang} tries to control a + \gls{ssr} client (with previously polling it)} + \label{lst:ssr-tests-sclang-controls-client-polled} + \end{listing}\\ + + In all tests for robustness, the attempt is made to break the + implementation of the \nameref{subsubsec:message_interface}. This is + achieved by purposely using ranges of data types for messages, that are + not allowed, or not defined in the \gls{ssr}'s internal + implementation.\\ + Two examples for weak spot exploitations are the use of negative + integers for \glspl{id} in source related messages (only non-zero, + non-negative \glspl{id} are allowed internally) or supplying an empty + string as hostname or port number for subscription messages.\\ + The first example will lead to undefined behavior, if the range is not + checked in the implementation, as a \textit{static\_cast} is used + internally to cast the value of the message data type (\textit{unsigned + int}) to the one expected by the \gls{ssr}'s Controller implementation + (\textit{signed int}) and the outcome of said operation is + implementation dependant (depending on the \gls{os} in use).\\ + The second example, if not checked for empty string, would lead to the + \gls{osc} interface trying to create a possibly defective address and + send poll messages out to it.\\ + While only some of the above mentioned scenarios could lead to a crash + of the program (under certain circumstances), unhandled all of them + waste ressources, which is undesired. + To circumvent possibly harmful input using the \gls{osc} interface, a + set of sanity checks were implemented, that only allow for a received + message to be processed, if all of its components fit the requirements. + + \subsubsection{Functionality and operability} + \label{subsubsec:functionality_and_operability} + + \begin{listing}[!htb] + \begin{mdframed} + \inputminted[numbers=left, firstline=216, lastline=225, + fontsize=\footnotesize]{supercollider} + {../../ssr/supercollider/tests.scd} + \end{mdframed} + \caption{supercollider/tests.scd: \gls{sclang} controls a \gls{ssr} + client (with previously polling it), creating several sources and + moving them} + \label{lst:ssr-tests-sclang-sources} + \end{listing}\\ + The test described in Listing~\ref{lst:ssr-tests-sclang-sources} is a + test for functionality, which also serves as a long-running stress test + for the \gls{ssr}. It creates 20 sources, that are then moved around + randomly, every 100ms, for 100 seconds, which on a Lenovo W540, with an + Intel i7-4700MQ and 16Gb RAM created less than 50\% of \gls{cpu} + load.\\ + Based on the above mentioned tests, the basic functionality of the + \gls{osc} interface can be guaranteed and depending on the host's + hardware also a maximum degree of capacity utilization can be + estimated, when observing the \gls{ssr}'s workload towards the system, + while using the long-running tests.\\ + It has to be mentioned, that a higher load can be observed, when using + higher levels of verbosity (especially above \textit{-vv}). This is + explained by the fact, that the \gls{ssr} will print out every + \gls{osc} message received and sent above the aforementioned verbosity + level. \cleardoublepage \section{Discussion} |