From 09c60958f46e4fbeaa1a8634971e772a559c1984 Mon Sep 17 00:00:00 2001 From: David Runge Date: Mon, 24 Jul 2017 12:01:23 +0200 Subject: thesis/thesis.tex: Adding bug fixes. Thanks Sabine. --- thesis/thesis.tex | 195 +++++++++++++++++++++++++++--------------------------- 1 file changed, 99 insertions(+), 96 deletions(-) (limited to 'thesis') diff --git a/thesis/thesis.tex b/thesis/thesis.tex index 59bc109..3a579a4 100644 --- a/thesis/thesis.tex +++ b/thesis/thesis.tex @@ -316,7 +316,7 @@ parskip=never]{paper} \label{sec:introduction} From the early days of stereo audio reproduction onwards, different kinds of spatial audio reproduction techniques have been developed and - established. Leading from plain stereophony to three-dimensional, + established, ranging from plain stereophony to three-dimensional, multi-channel setups. Their applications range from research to artistic and conventionally commercial fields, such as cinema and home entertainment.\\ @@ -324,10 +324,10 @@ parskip=never]{paper} With the rise of dynamic two and three-dimensional rendering algorithms (see~\ref{subsec:spatialaudiorenderingalgorithms}), the need for specialized software, implementing them, grew. Opposed to encoding of - spatial information of sources in only two channels --- static in the case - of commercially produced audio for radio and film --- encoding for massive - multi loudspeaker systems would not be feasible, when done statically, or - not applicable in the case of dynamic setups, reacting to user input in + spatial information of sources in only two channels (static in the case of + commercially produced audio for radio and film) encoding for massive multi + loudspeaker systems would not be feasible, when done statically, or not + applicable in the case of dynamic setups, reacting to user input in realtime.\\ Early dedicated hardware implementations, such as the \textit{Halaphon}, @@ -651,14 +651,15 @@ parskip=never]{paper} scientific context it can happen, that software is conceived by an institution, put to use, but later lacks the developers to drive the project onwards (e.g. \nameref{subsec:swonder}). - This way a high responsibility weighs on these institutions, as they + Therefore, a high responsibility weighs on these institutions, as they need to ensure further development on systems, not easily accessible to the public, or not feasible for home use (e.g. \gls{wfs}). This situation however also holds a great opportunity for coorporation.\\ As the development of free and open-source software is driven by its - users and its contributors, its main goal should at some point be to - build a large and dedicated community, so that not only bugs in the - source code can get fixed, but also new features be developed.\\ + users and its contributors, its main goal should be to build a large and + dedicated community at some point. Only this way new features can be + developed, while taking care of bugs in the already existing source + code.\\ Extending a software's functionality and improving its usability, such as that of the \gls{ssr}, can therefore be seen as an important step towards a more diverse user base and by that ensuring its further development. @@ -671,7 +672,7 @@ parskip=never]{paper} The application was chosen to be extended by a \gls{osc} based networking interface, because it runs on multiple \glspl{os}, offers a wide set of rendering algorithms (in various stages of completion) by using the - \gls{apf} \citep{MatthiasGeierTorbenHohn1890}, is used extensively in + \gls{apf} \citep{inproceedings:geieretal2012}, is used extensively in scientific research, has the future possibility to run medium and large scale \gls{wfs} setups and was still actively maintained by its creators at the time of writing.\\ @@ -699,10 +700,10 @@ parskip=never]{paper} in diverse setup scenarios (see~\ref{subsubsec:setups}). Therefore not only classic server-client relationships (see~\ref{subsubsec:remote_controlling_a_server}), but also client-only - and local (see~\ref{subsubsec:remote_controlling_a_client}) setups had to - be taken account of. In addition the case of medium and - large scale loudspeaker based rendering setups and their specifics had to - be considered (see~\ref{subsubsec:rendering_on_dedicated_speakers}). + and local (see~\ref{subsubsec:remote_controlling_a_client}) setups have + to be taken account of. In addition, the case of medium and large scale + loudspeaker based rendering setups and their specifics have to be + considered (see~\ref{subsubsec:rendering_on_dedicated_speakers}). \subsubsection{Prelimenaries} \label{subsubsec:preliminaries} @@ -728,14 +729,14 @@ parskip=never]{paper} enabled kernels and building new versions of (spatialisation) software.\\ The hardware of the large scale setup at \gls{tu-berlin} in lecture - hall H0104 was being updated and unusable at the time of writing. In - the future it will however become a valuable candidate for testing of - the sought after \gls{ssr} features, as its setup involves no Dante + hall H0104 was being updated and unusable at the time of writing. + However, in the future it will become a valuable candidate for testing + of the sought after \gls{ssr} features, as its setup involves no Dante network, but is instead run by several rendering computers connected to \gls{madi} and \gls{adat} lightpipe enabled speaker systems.\\ - Although a \gls{wfs} setup for testing purposes was eligible, it - generally is not needed for implementing the features described in - the following sections and subsections, as they can be tested using two + Although a \gls{wfs} setup for testing purposes was eligible, it is + generally not required for implementing the features described in the + following sections and subsections, as they can be tested using two machines running Linux, \gls{jack} and a development version of the \gls{ssr}.\\ @@ -778,9 +779,9 @@ parskip=never]{paper} \subsection{Publisher/Subscriber interface} \label{subsec:publisher_subscriber_interface} - The \gls{ssr} internally uses a \gls{pubsub}, which is a design pattern, + The \gls{ssr} internally uses a \gls{pubsub}, which is a design pattern to implement control through and over several parts of its components.\\ - In \gls{oop} \gls{pubsub} --- also called observer, listener messaging + In \gls{oop}, \gls{pubsub} --- also called observer, listener messaging --- is usually comprised of a publisher class, handling the messages, without explicitely implementing how they will be used and a subscriber class, that allows for its implementations to subscribe to the messages @@ -842,11 +843,11 @@ parskip=never]{paper} port, called “\gls{ip} interface”. This has the benefit of reusing the same \gls{xml} parser code in use for scene and reproduction description.\\ - A downside is however, that --- from the perspective of other software - --- it is complicated to use, as a conversion to \gls{xml} has to be - attempted before sending a message to the \gls{ssr}. Additionally, the - message has to be linted (error checked) before sending and parsed again, - after receiving an answer from the application.\\ + From the perspective of other available software, it is a downside + though, that it is complicated to use, as a conversion to \gls{xml} has + to be attempted before sending a message to the \gls{ssr}. Additionally, + the message has to be linted (error checked) before sending and parsed + again, after receiving an answer from the application.\\ The \gls{ip} interface achieves to offer more or less direct access to the \nameref{subsec:publisher_subscriber_interface}. However, it has no notion of a networked setup and could therefore be described as a @@ -871,7 +872,7 @@ parskip=never]{paper} As mentioned in section~\ref{subsec:publisher_subscriber_interface}, the \mintinline{c++}{NetworkSubscriber} class (part of the \gls{ip} interface) implements the \mintinline{c++}{Subscriber} interface. This - means: The network interface subscribes to the messages the + implies that the network interface subscribes to the messages the \mintinline{c++}{Publisher} (the \mintinline{c++}{Controller} instance) has to offer. Every time a function of the \gls{ssr}'s \mintinline{c++}{Controller} instance, that was inherited from @@ -901,10 +902,10 @@ parskip=never]{paper} Initial attempts, such as the mapping of a networking setup in the scene description\footnote{ \href{https://github.com/dvzrv/ssr/tree/distributed\_reproduction} - {https://github.com/dvzrv/ssr/tree/distributed\_reproduction}} - proved too restrictive though, as it would allow the networking - functionality only to renderers, that use loudspeakers and mix scene - description with networking description.\\ + {https://github.com/dvzrv/ssr/tree/distributed\_reproduction}}, proved + too restrictive though, as it would allow the networking functionality + only to renderers, that use loudspeakers and mix scene description with + networking description.\\ A nearly configuration-less approach, based on subscribing clients on sending poll messages to them proved more open (in the sense that it can be interfaced with by any \gls{osc}-capable application or programming @@ -968,13 +969,12 @@ parskip=never]{paper} other multimedia devices” \citep{website:oscv1.0} developed at the \gls{cnmat}. Its 1.0 specification was published by Matthew Wright in 2002 \citep{website:oscv1.0} and the protocol has found widespread - implementations (as libraries) in several programming languages and - through that many use-cases in free and closed audio and video related - applications (e.g. Ardour \citep{website:ardour}, Max/MSP - \citep{website:cycling74}, \gls{supercollider} - \citep{website:supercollider}) since then.\\ \gls{osc}'s syntax is - defined by several parts, which are discussed briefly in this - section.\\ + implementations (as libraries) in several programming languages. + Many free and closed audio and video related applications (e.g. Ardour + \citep{website:ardour}, Max/MSP \citep{website:cycling74}, + \gls{supercollider} \citep{website:supercollider}) make use of it.\\ + \gls{osc}'s syntax is defined by several parts, which are discussed + briefly in this section.\\ \begin{itemize} \item Atomic data types, which are also reflected in type tags (see @@ -992,7 +992,8 @@ parskip=never]{paper} According to the specification, applications sending \gls{osc} packets are considered a client and the ones receiving packets a server. - Applications can therefore be client and server at the same time.\\ + Therefore, applications can both be client and server at the same + time.\\ \begin{table}[!htb] \renewcommand{\arraystretch}{1.2} @@ -1039,10 +1040,11 @@ parskip=never]{paper} As shown in Table~\ref{tab:ssr-osc-data-type-acronyms}, only \mintinline{c++}{int32}, \mintinline{c++}{float32}, \gls{osc}-string - and \gls{osc}-blob are considered standardized. Most of the remaining - non-standard types are however implemented and used by many different + and \gls{osc}-blob are considered standardized. However, most of the + remaining non-standard types are implemented and used by many different clients. For implementing the \gls{ssr} \gls{osc} interface, described - in the following subsections, it was necessary to use the + in subsection~\ref{subsubsec:liblo} + --~\ref{subsubsec:workflow_examples}, it was necessary to use the non-standard types \textit{True} and \textit{False} alongside the standard-types. @@ -1069,12 +1071,12 @@ parskip=never]{paper} class, at the core of the C++ side of the library, is responsible for assigning a network port to listen to for incoming messages, listening for messages, executing code on their arrival (i.e.\ callback handling) - and sending messages to clients. Many applications, facilitating - liblo, use \gls{osc} only as a messaging system. This usually means, - that such an application itself is not single-purpose and is busy - computing something else most of the time. Therefore it makes sense to - run a Server instance on a separate background thread, to not interfere - with the executional flow of the rest of the program.\\ + and sending messages to clients. Many applications facilitating liblo + use \gls{osc} only as a messaging system. This usually means, that such + an application itself is not single-purpose and is busy computing + something else most of the time. Therefore it makes sense to run a + Server instance on a separate background thread, to not interfere with + the executional flow of the rest of the program.\\ The \mintinline{c++}{ServerThread} class is able to free its ressources upon going ot of scope (i.e.\ their ressources are not used by any object or function anymore), known as \gls{raii}. For this reason, the @@ -1085,10 +1087,11 @@ parskip=never]{paper} \label{subsubsec:starting-the-ssr} The \gls{ssr} can be started with a rendering engine preselected (an executable postfixed by the supported rendering algorithm is provided - by the software --- e.g. \textbf{ssr-wfs}), or by selecting one through - the configuration file, when using the standard executable named - \textbf{ssr}. This way \gls{aap}, \gls{bs}, \gls{brs}, generic, - \gls{nfc-hoa}, \gls{vbap}, \gls{wfs} renderers become available.\\ + by the software bundle --- e.g. \textbf{ssr-wfs}) or by selecting one + through the configuration file, when using the standard executable + named \textbf{ssr}. This way, the following renderers become available: + \gls{aap}, \gls{bs}, \gls{brs}, generic, \gls{nfc-hoa}, \gls{vbap} and + \gls{wfs}.\\ Additional features can be activated with the help of several flags to the executables. The customized ones, belonging to the \gls{osc} interface will be discussed in the following subsections. More @@ -1126,9 +1129,9 @@ ssr-binaural -p “50002” \paragraph{Server instance} \label{para:server-instance} With the help of the \textbf{-N} flag, it is possible to start the - \gls{ssr} as an \gls{osc} server. For additional future purposes of - the flag, such as a \gls{gui} only mode, - see~\ref{subsubsec:non-renderer}. In + \gls{ssr} as an \gls{osc} server. Additionally, the flag can be used + in a future extension of the networking interface + (see~\ref{subsubsec:non-renderer}). In Listing~\ref{lst:ssr-binaural-server-start} additionally, flag \textbf{-C} is used to specify an initial client \gls{ip} and its port (the flag actually accepts a comma-separated list of @@ -1424,23 +1427,23 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \subsubsection{Message interface} \label{subsubsec:message_interface} - \gls{osc} offers the possibility of a hierarchical path tree, that can + \gls{osc} offers the possibility of a hierarchical path tree that can be used to group messages by type (i.e.\ context). In conjunction with messages only understood by client or server (or a context dependant meaning), most of the messages understood by the \nameref{subsec:ip-interface} are implemented. Additional features, related to server-client and client-only functionality, were integrated as well.\\ - In general, it can be distinguished between \textit{direct} messages - - sent from a server (or an application mimicking one) to a client or a - server to trigger processing (see + In general, it can be distinguished between \textit{direct} messages + --- sent from a server (or an application mimicking one) to a client or + a server to trigger processing (see Table~\ref{tab:ssr-osc-processing-tracker-transport}), reference (see Table~\ref{tab:ssr-osc-reference}), scene (see Table~\ref{tab:ssr-osc-scene}), source (see Table~\ref{tab:ssr-osc-source}), tracker (see Table~\ref{tab:ssr-osc-processing-tracker-transport}) or transport (see Table~\ref{tab:ssr-osc-processing-tracker-transport}) related - operations in the \gls{ssr} --- and \textit{update} messages (see + operations in the \gls{ssr} and \textit{update} messages (see Table~\ref{tab:ssr-osc-update}) --- sent from a client to a server upon successful processing an operation related to a \textit{direct} message.\\ @@ -1855,8 +1858,8 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \section{Discussion} \label{sec:discussion} The \gls{osc} based networking extension created for the \gls{ssr} can be - considered a valuable usability improvement. Its implemented features are - discussed further in the following section, followed by an outlook on + considered as a valuable usability improvement. Its implemented features + are further discussed in the following section, followed by an outlook on related future work. The extension is additionally extensively documented in the source code, to ensure the ease of further development.\\ Due to the versatility of how the \gls{ssr} can be used in a networking @@ -1885,7 +1888,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” server-only and messages available to clients and servers alike (see~\ref{subsubsec:message_interface}). Examples for different workflows are given in~\ref{subsubsec:workflow_examples} to illustrate simple - use-cases.\\ + use cases.\\ This puts the \gls{osc} interface in the unique position of providing a native messaging interface and a flexible architecture. It can be used from single local instances up to large scale networked setups (with the @@ -1904,16 +1907,17 @@ ssr-aap -N “server” -C “127.0.0.1:50002” lowers the complexity of the network topology (\gls{udp} does not perform handshakes for every packet sent, unlike \gls{tcp}) and thus the size of each message sent.\\ - The \gls{osc} interface therefore implements messaging, using lower - bandwidth, at a greater feature set. In~\ref{subsec:automated_tests} a - test environment is introduced, that further elaborates on the overall - functionality and feasibility of the message interface. + The \gls{osc} interface therefore implements messaging, while using lower + bandwidth and offering a greater feature set. + In~\ref{subsec:automated_tests} a test environment is introduced, that + further elaborates on the overall functionality and feasibility of the + message interface. \subsection{Automated tests} \label{subsec:automated_tests} The \gls{ssr} was developed without the help of a test framework, which is responsible to test its components, after they have been changed. - This means, that internal (e.g.\ the \gls{pubsub} interface), or external + This means, that internal (e.g.\ the \gls{pubsub} interface) or external (e.g.\ the \gls{ip} or \gls{osc} interface) functionality might or might not work as expected. To test the \gls{osc} interface's logical coherency and robustness automatically, a set of tests was written in @@ -1930,9 +1934,9 @@ ssr-aap -N “server” -C “127.0.0.1:50002” Listing~\ref{lst:ssr-tests-sclang-unsubscribed-controls-server} and~\ref{lst:ssr-tests-sclang-subscribed-controls-server} describe - server-side tests for robustness. While the first will not lead to any - processed action by the server, the latter will. This is explained by - \gls{sclang} not being a subscribed client with a + server-side tests for robustness. While the first test will not lead to + any processed action by the server, the latter will. This is explained + by \gls{sclang} not being a subscribed client with a \mintinline{c++}{MessageLevel} of \mintinline{c++}{SERVER} or higher in the first case. However, in the second test \gls{sclang} subscribes to the \gls{ssr} server instance, which is why the \gls{osc} messages are @@ -1964,8 +1968,8 @@ ssr-aap -N “server” -C “127.0.0.1:50002” Listing~\ref{lst:ssr-tests-sclang-controls-client-unpolled} and~\ref{lst:ssr-tests-sclang-controls-client-polled} are client-side tests for robustness, that work in a similar fashion to the - aforementioned server-side tests. While in the first case, the sent - \gls{osc} messages are not evaluated, because \gls{sclang}, mimicking a + aforementioned server-side tests. While the sent \gls{osc} messages are + not evaluated in the first case, because \gls{sclang}, mimicking a server instance (see~\ref{para:server_mimicry}), did not poll the \gls{ssr} client instance up front, in the second case the messages are evaluated, because it did poll the client first. @@ -1992,10 +1996,10 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \label{lst:ssr-tests-sclang-controls-client-polled} \end{listing}\\ - In all tests for robustness, the attempt is made to force errors in the + In all tests for robustness the attempt is made to force errors in the implementation of the \nameref{subsubsec:message_interface}. This is achieved by purposely using ranges of data types for messages, that are - not allowed, or not defined in the \gls{ssr}'s internal + not allowed or not defined in the \gls{ssr}'s internal implementation.\\ Two examples for weak spot exploitations were the use of negative integers for \glspl{id} in source related messages (only non-zero, @@ -2007,7 +2011,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” int}) to the one expected by the \gls{ssr}'s Controller implementation (\textit{signed int}) and the outcome of said operation is implementation dependant (depending on the \gls{os} in use).\\ - The second example, if not checked for empty string, would lead to the + The second example, if not checked for empty string, will lead to the \gls{osc} interface trying to create a possibly defective address and send poll messages out to it.\\ While only some of the above mentioned scenarios could lead to a crash @@ -2035,9 +2039,8 @@ ssr-aap -N “server” -C “127.0.0.1:50002” The test described in Listing~\ref{lst:ssr-tests-sclang-sources} is a test for functionality, which also serves as a long-running stress test for the \gls{ssr}. It creates 20 sources, that are then moved around - randomly, every 100ms, for 100 seconds, which on a Lenovo W540, with an - Intel i7-4700MQ and 16Gb RAM created less than 50\% of \gls{cpu} - load.\\ + randomly, every 100ms, for 100s, which on a Lenovo W540, with an Intel + i7-4700MQ and 16Gb RAM created less than 50\% of \gls{cpu} load.\\ Based on the above mentioned tests, the basic functionality of the \gls{osc} interface can be guaranteed and depending on the host's hardware also a maximum degree of capacity utilization can be @@ -2051,7 +2054,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \subsection{Future Work} \label{subsec:future_work} - Several features, interesting for different use-cases, were out of scope + Several features, interesting for different use cases, were out of scope for this work. They are however complementary to the \gls{osc} networking extension, or can be implemented on top of it and will be discussed in the following sections.\\ @@ -2099,7 +2102,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” be imagined as a \gls{gui} only frontend.\\ Figure~\ref{fig:ssr-client-server-clients-only-shared-output} illustrates a scenario, in which a server instance is used to control a - set of \textit{n} clients, that collectively render audio (e.g.\ on a + set of \textit{n} clients, that collectively renders audio (e.g.\ on a large scale \gls{wfs} or \gls{nfc-hoa} system). In contrast to the client instances, the server does not render any audio (i.e.\ has no outputs) and might not even need any audio input.\\ @@ -2144,7 +2147,7 @@ ssr-aap -N “server” -C “127.0.0.1:50002” capable application), controlling the server in a setup, similar to the one shown in Figure~\ref{fig:ssr-external-client-server-separate-output}.\\ - For the aforementioned use-cases to work, the \gls{gui} of the + For the aforementioned use cases to work, the \gls{gui} of the \gls{ssr} has to be extended to show the networking specific information and be able to use the \gls{osc} interface through the \gls{pubsub} interface (which itself would probably have to be extended @@ -2204,14 +2207,13 @@ ssr-aap -N “server” -C “127.0.0.1:50002” attribute for each part of the setup, that is referencing a loudspeaker --- the hostname or \gls{ip} address of the host --- by extending the \gls{asdf}.\\ - More work has to be put into implementing this feature, or rather - improvement however, as it also requires tests in medium and large - scale setups, to ensure a discrete rendering, as if only using one - host. + However, more work has to be put into implementing this feature, or + rather improvement, as it also requires tests in medium and large scale + setups, to ensure a discrete rendering, as if only using one host. \subsubsection{Status messages} \label{subsubsec:status_messages} - When reflecting about different use-cases for networking setups + When reflecting about different use cases for networking setups involving the \gls{ssr}, it became apparent, that in certain situations it would be desirable to be able to poll instances for information, involving sources, scenes and the like.\\ @@ -2219,8 +2221,8 @@ ssr-aap -N “server” -C “127.0.0.1:50002” \gls{gui} (e.g.\ non-interactive display of source positions) in another programming language, such as \gls{python}, \gls{pd}, or \gls{supercollider}, while only relying on \gls{osc} for communication - between the parts. Another would be the case, where certain aspects of - a client or server instance should be monitored (e.g.\ \gls{cpu} + between the parts. Another example is the implementation of monitoring + of certain aspects of a client or server instance (e.g.\ \gls{cpu} usage). Both examples should allow a \gls{gui} (or any other monitoring) process to be subscribed, after the active \gls{ssr} instances started rendering.\\ @@ -2308,7 +2310,8 @@ ssr-aap -N “server” -C “127.0.0.1:50002” of scenes and reproduction setups alike. This becomes especially useful in the case of listening experiments, as the experiments will not have to rely on a mixture of different scripting languages anymore (e.g.\ - see experimental setup in attachment of \citep{DmitryGrigoriev1896}). + see experimental setup in attachment of + \citet{mastersthesis:grigoriev2017}). \subsubsection{Interpolation of moving sources} \label{subsubsec:interpolation_of_moving_sources} @@ -2394,11 +2397,11 @@ ssr-aap -N “server” -C “127.0.0.1:50002” InertiaCube3).\\ In the specific case of setting up a large array of independent \gls{brs} or \gls{bs} renderers, connected to one server instance or - application, it might however be needed to extend the messaging system - to allow passing on of messages from one client to a server only - towards one specific other client (a type of proxy messaging). This - would ensure, that every renderer can be supplied with a specific - stream of \gls{osc} messages from its assigned head tracker. Single + application, it might be required to extend the messaging system to + allow passing on of messages from one client to a server only towards + one specific other client (a type of proxy messaging). This would + ensure, that every renderer can be supplied with a specific stream of + \gls{osc} messages from its assigned head tracker. Additionally, single (and local) renderers can be started as a server instance and clients can be assigned to them flexibly. -- cgit v1.2.3-70-g09d2