\chapter{Design and Implementation}\label{chap:design}
This chapter outlines and further describes in detail the design decisions behind the end-product of this project, as well as the specifics on the implementation of the project.

\section{Principles of Operation}
The analysis of volume flow can be broken down into two fundamental operations that must be carried out:
\begin{itemize}
\item A calculation of the \textbf{cross-sectional area} of a slice (or slices) of material
\item A calculation of the \textbf{velocity} of the material flow
\end{itemize}

\subsection{Cross-Sectional Area}

The methodology used in order to analyze the cross-sectional area of the material flow is \textbf{geometric analysis}. Simply put, the geometry of a laden belt is compared with that of an empty belt. The resulting difference in area is that of the material itself.

In order to accomplish this analysis, a horizontal slice of the sensor data is used---see \autoref{fig:conveyor_top}. The slice represents the depth data of a single dimension, in this case, the crosswise dimension of the belt.

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{design/conveyor_top}
\caption{Graphical depiction of the LIDAR sensor image. The slice is a one-dimensional extract of the sensor image crosswise over the belt.}
\label{fig:conveyor_top}
\end{figure}

\begin{figure}[h]
	\centering
    \includegraphics[width=0.6\textwidth]{design/cross_analysis_new}
    \caption{Cross-section of material flow on a conveyor belt.}
    \label{fig:cross_analysis}
\end{figure}

During calibration, the empty belt is used to fit the polynomial belt curve $f(x)$. The fitting of this nth-degree polynomial is done with the least-squares method.

After calibration, the current slice curve $g(x)$ can be used to obtain the Cross-Sectional Area $A_C$ as shown in \autoref{eq:cross_area} and \autoref{fig:cross_analysis}. 

\begin{equation} 
  A_C = \int^{x_b}_{x_a}\left[ g(x) - f(x) \right] dx \label{eq:cross_area}
\end{equation}

\subsubsection{Further Considerations}
The accuracy of the computed cross-sectional area depends primarily on the accuracy of the depth data as well the frame rate of the sensor.

\begin{minipage}{\textwidth}
However, further operations may be implemented in order to increase accuracy, such as:
\begin{itemize}
\item Computing the cross-sectional area from multiple slices of each frame and averaging these
\item Computing the average cross-sectional area between frames, in order to create a smoother---and possibly more accurate---estimation of the volume flow
\end{itemize}
\end{minipage}

It is important to note though, that the implementation of further operations may exhaust the processing capabilities of the platform. Therefore, a crucial balance must be struck between performance and accuracy.

\subsection{Belt Velocity}
Conventional belt scales use some form of a rotary encoder in order to measure the belt velocity. This is---however accurate---only an approximation of the velocity of the material flow itself, since material velocity may deviate from belt velocity depending on environmental or material conditions.

For the purposes of this project, the term \textit{belt velocity} will be used loosely to describe an indeterminate aggregate of belt, volume and surface velocities. This is because the techniques used in order to determine the velocity---as shown below---are not designed to isolate a single velocity type.

The fundamental operation being used in the following methods in order to determine the belt velocity is \textbf{cross-correlation}. In essence, a signal is compared to an older version of itself in order to determine a displacement---or in signal terms, a \textit{delay}.

In the case of this project, given the known interval between two consecutive signals---i.e.\ the frame rate---it is possible to express this delay in the form of a physical displacement, in meters.

The various analytic methods used in this project differ only by which data is selected to represent the signal during cross-correlation. The algorithm of calculating the cross-correlation itself remains the same.

Equation \ref{eq:cross_corr} shows how the cross-correlation $r$ between two signals $A$ and $B$ of lengths $n$ may be calculated by multiplying each element $i$ together. This is done for each possible delay $d$ value. The maximum value of the correlation $r$ corresponds to the most likely value of $d$.

\begin{equation}
r(d) = \sum_{i=1}^{n} A_i \cdot B_{i+d} \label{eq:cross_corr}
\end{equation}

\subsubsection{Chosen Method - Statistical Method}
This method was developed as an aggregate of the previously attempted methods, improving on and solving issues earlier iterations had. It is therefore simply the most successful iteration.

The statistical method carries out the following operations:
\begin{enumerate}
\item A user-provided area of interest is cropped out of the entire sensor frame. This is done to isolate only the most relevant and data-dense regions, as well as to eliminate error from static elements as much as possible.
\item This subset of the frame is then divided into one-dimensional vertical strips.
\item For each of the strips, the cross-correlation displacement is calculated.
\item With the of displacement values for each strip, statistical outlier values are removed and a mean displacement is calculated.
\item This mean displacement in pixels, together with camera frame geometry, is used to calculate the physical displacement in meters.
\end{enumerate}

The use of this statistical approach using multiple vertical strips---see \autoref{fig:conveyor_xcorr}---is very similar to directly using 2-dimensional correlation, however it attempts to solve a significant problem with the 2-dimensional cross-correlation method.

\begin{figure}[h]
\centering
\includegraphics[width=0.8\textwidth]{design/conveyor_xcorr}
\caption{Representation of the area of interest and vertical strips used for the cross-correlation operations.}
\label{fig:conveyor_xcorr}
\end{figure}

Since a 2-dimension cross-correlation would simultaneously consider the entire area of interest, any static elements in the frame would highly influence the results of the correlation, causing it to always be close to zero. This introduces a high error and variability in the result.

This statistical approach allows us to discard outlier values---such as values close to zero---and retain only those slices which do not contain any static elements.

\subsubsection{Alternative Methods}

Before arriving at the statistical method described above, multiple iterations of possible methods were tested. In this subsection, two of the most significant alternative methods are described.

Firstly, as already mentioned above, the \textbf{2-dimensional cross-correlation}. This method produces robust values and is less computationally complex than the statistical approach, however it is significantly more sensitive to static elements. This introduces many challenges since static elements may not be entirely avoided, either on the belt, or on the sensor itself.

The other alternative method, called the \textbf{topographical method}, is much less computationally expensive, since it only runs one cycle of the cross-correlation algorithm per frame.

The topographical approach works in the following manner. The values within each \textbf{crosswise} slice are summed. This reduces the 2-dimensional sensor data into a 1-dimensional representation which is called the \textit{topography}. This topography can be used as the signal for cross-correlation.

Although this method was robust to signal noise, the uncertainty of the values was due to information being lost in the summation process.


\subsection{Volume Flow}

Once the cross-sectional area and velocity of the material have been determined, the only operation that remains is to derive from them the volume flow.

The volume of material that has passed the sensor per frame $V_F$, can be calculated simply as described in Equation \ref{eq:volume_flow}. $A_{C}$ is the cross-sectional area measured in a particular frame, $v_m$ is the velocity of the material flow and $f$ is simply the framerate of the sensor.

\begin{equation}
  V_F =  A_{C} \cdot v_m \cdot \frac{1}{f} \label{eq:volume_flow}
  % V_F &= \left( \frac{A_{C,F} + A_{C,F-1}}{2}\right) \cdot v_m \cdot
  % \frac{1}{f} \label{eq:volavg}
\end{equation}

\subsection{Accuracy}

The accuracy of the system is limited primarily by the framerate of the sensor.

Figure \ref{eq:accuracyupperlimit} shows the theoretical maximum accuracy for a simple single line-scanner method of determining volume. Thus, for the targeted framerate of 30 FPS, the theoretical maximum accuracy of this method is limited to \SI{3.3}{\percent}.

\begin{equation}\label{eq:accuracyupperlimit}
\text{Accuracy Upper Limit} = \frac{1}{f} \cdot \SI{100}{\percent}
\end{equation}

Introducing multiple line-scans per frame would proportionally reduce this upper limit, at the cost of increased computational complexity.

The various manufacturers of conventional belt scales have claims of accuracy between \SI{0.5}{\percent} and \SI{2}{\percent}.


\section{Phases of Development}\label{sec:developmentphases}
The following phases of development are not grouped chronologically over the span of the project schedule, rather into conceptual groups.

\subsection{Preparing Development and Build Environment}
Setting up the development environment for this project was a non-trivial task, due to a lack of pre-compiled binaries for the intended architecture, the ARM chipset of the Raspberry Pi.

The Raspberry Pi 4 Model B used in this project was delivered with a Quad Core Cortex-A72 64-bit SOC. However, a 32-bit kernel OS was used in this project, due to the netHAT drivers being delivered as 32-bit compiled binaries.

\subsubsection{Intel RealSense SDK}
The Intel RealSense SDK, or librealsense, is a cross-platform library provided by Intel for use with their RealSense devices. Pre-compiled binaries for 32-bit ARM were not provided, and therefore must be compiled \cite{realsense_git}.

\subsubsection{ZeroMQ}
ZeroMQ is a lightweight asynchronous messaging library that extends the traditional socket interfaces \cite{zeromq_git}. In this project, ZeroMQ was used in order to:
\begin{itemize}
\item Broadcast raw sensor data to remote controllers using a publish/subscribe model
\item Exchange configuration parameters between the remote controller and the local processor using a request/reply model
\end{itemize}

In this project, TCP sockets were used for communications.

ZeroMQ was chosen to provide the aforementioned functionalities for the following reasons:
\begin{itemize}
\item Simple to implement
\item Data can be transferred as binary data, instead of requiring serialization
\item Availability on numerous platforms, as well as pre-compiled binaries
\end{itemize}

\subsubsection{Real-Time Kernel Patch}
Traditionally, the Linux kernel only allows one process to preempt another process in specific circumstances. This means, that even a high-priority thread may not preempt kernel code until the kernel explicitly yields control.

This is particularly disadvantageous for any operations requiring real-time performance. In order to circumvent this, the \verb|CONFIG_PREEMPT_RT| Kernel Patch is used in order to allow kernel code to be preempted \cite{rtwiki}.

In the case of this project, this means that the local processor can process and deliver data in a more deterministic fashion.

\subsubsection{GUI with Qt}
The Qt GUI framework \cite{qtWebsite} was used in order to create a GUI for the remote controller. This allowed for the sensor data to be more easily calibrated and aligned, as well as providing a consistent interface for end-user configuration. Qt was chosen for its ease of use, as well as its ability to be compiled cross-platform.

\subsection{Development of Main Functionality}
At this stage of the design process, the functionality that is fundamental to the principle operation described earlier was developed. These functions include:
\begin{itemize}
\item Transmission of raw sensor data
\item Calibration of sensor data
\item Configuration for processing
\item Transmission of configuration parameters
\end{itemize}

\subsection{Testing and Validation}
The project was largely developed in an iterative process with multiple cycles of testing and validation. This allowed more fundamental elements to be developed and debugged first, before building more complex elements on top.

The iterative cycles can be broken down into:
\begin{enumerate}
\item \textbf{Sandbox Testing:} At this stage, the most fundamental features were developed and tested, such as:
\begin{enumerate}
\item Connection to LIDAR sensor and retrieval of data
\item Representing the data visually
\item Basic manipulation of the sensor data
\end{enumerate}
\item \textbf{Laboratory Testing:} In the lab, the main features and functionalities of the program could be developed and tested on a more controlled, smaller scale. Among the main functionalities:
\begin{enumerate}
\item Development of the processing algorithms, i.e.\ for cross-correlation, determining the cross-sectional area
\item Processing of output data
\item Establishing and transmitting output data over Profinet
\item Tuning performance
\end{enumerate}
\item \textbf{Field Testing:} At a certain point, development in the laboratory setting could no longer proceed without taking into consideration the environmental and full-scale factors of deployment in the field. Thus, the final iteration of development must be carried out on-site.
\end{enumerate}

See \autoref{chap:validation} for elaboration and results of each of the testing cycles.

\section{Components}\label{sec:componenets}

As already touched upon in \autoref{sec:aims}, the components used in this project were chosen mainly for their commercial availability and low cost. This section will elaborate more on the decision to select these specific components.

The cost for each of these components are listed at the end of this section in \autoref{table:cost}.

\textbf{Raspberry Pi 4 Model B}\nopagebreak

The Raspberry Pi was chosen as the computation platform primarily for its widespread use in IoT and IIoT, low cost and commercial availability. It also supports the Linux kernel and operating system which greatly eases the software development and deployment process. 

As shown in \autoref{tab:rpi}, the Quad-Core ARM processor as well as 2 GB memory capacity provide ample performance for the intended computation. The wireless networking capability of the Raspberry Pi makes it an ideal candidate for an IoT product.

\begin{table}[H]
\centering
\begin{tabularx}{0.75\textwidth}{| c | >{\centering\arraybackslash}X |}
\hline
Processor & Quad Core Cortex-A72 (ARM v8) 64-Bit \\
\hline
Memory & 2GB \\
\hline
\multirow{3}*{Networking} & 2GHz and 5GHz 802.11ac Wireless \\
& Bluetooth 5.0 \\
& Gigabit Ethernet \\
\hline
\multirow{3}*{Connectivity} & 2x USB 3.0 \\
& 2x USB 2.0 \\
& 40 pin GPIO header \\
\hline
Storage & Micro-SD card slot \\
\hline
Operational Temperature & \SI{0}{\celsius} to \SI{50}{\celsius} \\
\hline
\end{tabularx}
\caption{The relevant technical specifications of the Raspberry Pi 4 Model B used in this project \cite{rpiSpecs}.}
\label{tab:rpi}
\end{table}


\textbf{Intel RealSense L515}\nopagebreak

The Intel Realsense L515 was also chosen primarily for its low cost. However, the open-source and Linux-friendly nature of Intel's RealSense SDK also make it an ideal choice to pair with the Raspberry Pi. The small form-factor of the sensor would also allow a final product size that would be compact and easy to install.

\autoref{table:l515} provides an overview of the specifications of the RealSense L515 sensor. 

\begin{table}[H]
\centering
\begin{tabularx}{0.75\textwidth}{ >{\centering\arraybackslash}X  | >{\centering\arraybackslash}X | >{\centering\arraybackslash}X |}
\cline{2-3}
& Depth & Color\\
\hline
\multicolumn{1}{|c|}{Resolution} & Up to 1024x768 & Up to 1920x1080\\
\hline
\multicolumn{1}{|c|}{Field-of-View} & \ang{70}x\ang{55} & \ang{69}x\ang{42} \\
\hline
\multicolumn{1}{|c|}{Frame Rate} & \multicolumn{2}{c|}{30 Frames per Second}\\
\hline
\multicolumn{1}{|c|}{ \multirow{2}*{Depth Range} } & \multicolumn{2}{c|}{Up to \SI{3.9}{\meter} at 15\% Reflectivity} \\
\multicolumn{1}{|c|}{} & \multicolumn{2}{c|}{Up to \SI{9}{\meter} at 95\% Reflectivity}\\
\hline
\multicolumn{1}{|c|}{ \multirow{2}*{Depth Accuracy} } & \multicolumn{2}{c|}{\SI{5}{\milli\meter} at \SI{1}{\meter}}\\
\multicolumn{1}{|c|}{} & \multicolumn{2}{c|}{\SI{14}{\milli\meter} at \SI{9}{\meter}}\\
\hline
\multicolumn{1}{|c|}{Laser Wavelength} & \multicolumn{2}{c|}{\SI{860}{\nano\meter} Infrared}\\
\hline
\multicolumn{1}{|c|}{Power Consumption} & \multicolumn{2}{c|}{Up to \SI{3.3}{\watt}}\\
\hline
\multicolumn{1}{|c|}{Operational Temperature} & \multicolumn{2}{c|}{\SI{0}{\celsius} to \SI{30}{\celsius}}\\
\hline
\multicolumn{1}{|c|}{\multirow{2}*{Mounting Options}} & \multicolumn{2}{c|}{ISO1222 Tripod Mounting Point}\\
\multicolumn{1}{|c|}{} & \multicolumn{2}{c|}{2x M3 Mounting Point}\\
\hline
\end{tabularx}
\caption{The relevant technical specifications of the Intel RealSense L515 used in this project \cite{realsenseDatasheet}.}
\label{table:l515}
\end{table}

\textbf{netHAT}\nopagebreak

The netHAT by Hilscher provides a simple-to-use Industrial Ethernet interface for the Raspberry Pi. Through the Raspberry Pi HAT standard, the netHAT is easily installed on the GPIO pins of the Raspberry Pi.

The drivers---called CIFX---and API library provided--called libCIFX---provide a simple way to interface with Industrial Ethernet networks from software. Section \ref{sec:softarch} shows how CIFX was integrated into the rest of the software architecture.

\autoref{table:nethat} gives an overview of the capabilities and specifications of the netHAT.

\begin{table}[H]
\centering
\begin{tabularx}{0.75\textwidth}{ | c | >{\centering\arraybackslash}X | } \hline
Processor & Hilscher netX 52 \\ \hline
Memory & 4MB Quad SPI Flash \\ \hline
Interface & SPI up to 125MHz \\ \hline
Network & 2x Ethernet 100 BASE-TX \\ \hline
\end{tabularx}
\caption{The relevant technical specifications of the Hilscher netHat \cite{nethatHilscher}.}
\label{table:nethat}
\end{table}

\textbf{Cost Breakdown}\nopagebreak

\autoref{table:cost} lists the individual costs of each of the components, and their total. This total is not reflective of the final cost of the completed product as it does not yet include costing for the housing, wiring and other installation costs.

\begin{table}[H]
\centering
\begin{tabular}{| c | c |}
\hline
Intel RealSense L515 & \euro{380}\tablefootnote{The cost has since increased to \euro{570} as of January 2022.}\\ \hline
Raspberry Pi Model 4 B & \euro{84} \\ \hline
netHAT & \euro{69} \\ \hline
\textbf{Total} & \textbf{\euro{553}\tablefootnote{Due to a malfunctioning unit, a new sensor was purchased, bringing the total cost of development to around \euro{1120}.}} \\ \hline
\end{tabular}
\caption{Breakdown of the costs used in the development of this project.}
\label{table:cost}
\end{table}

\section{Process Overview}
With the objective of creating a marketable commercial product in mind, the process flow was designed for ease-of-use and ease-of-configuration for the end-user. This is the justification for implementing a remote controller that allows the setup to be remotely configured once installed.

While \autoref{fig:processoverview} gives a brief overview of the interrelationship of the remote and local sides in the complete process, it is here further elaborated:
\begin{enumerate}

\item \textbf{Transmission of Raw Data:} Upon the first startup, the local processing software immediately begins to broadcast the raw sensor data. This data can then be subscribed to by a remote controller.

\item \textbf{Remote Configuration:} Once the remote devices have subscribed to the raw data broadcast, it will be previewed on the FlowRemote GUI interface. See \autoref{fig:flowremote} for an overview of the FlowRemote interface.
\begin{enumerate}
\item \textbf{Image Pre-Processing:} The engineer then pre-processes the image---rotating, skewing and cropping---until the conveyor belt appears vertically aligned and the perspective has been corrected.
\item \textbf{Calibration and Fitting:} Now the engineer may select a single layer or slice of the sensor image which will be used to fit for the conveyor belt curve. The fitting parameters are selected, and the calibration is saved. This corresponds to the curve $f(x)$.
\item \textbf{Belt Parameters:} In order to correctly determine the belt velocity and volume flow, the conveyor belt parameters such as visible length and width need to be provided.
\end{enumerate}

\item \textbf{Transmission of Configuration Parameters:} The parameters that were configured in the previous step are then transmitted back to the local processor, FlowPi, and local processing mode is then engaged.

\item \textbf{Local Processing:} In local processing mode, raw data is no longer transmitted for performance purposes. The sensor data is directly processed on the Raspberry Pi, and the outputs are delivered over Profinet to the PLC.

\end{enumerate}

\begin{figure}[h]
	\centering
	\includegraphics[width=0.8\textwidth]{./design/ProcessOverview}
	\caption{Overview of the communication and processing process between the remote controller and the local processor.}
	\label{fig:processoverview}
\end{figure}

\section{Software Architecture}\label{sec:softarch}

The software architecture developed in this project---see \autoref{fig:SoftwareArchitecture}---consists of two separate but tightly interconnected parts, namely:
\begin{itemize}
\item \textbf{FlowPi:} The local processing software that runs on the Raspberry Pi, and
\item \textbf{FlowRemote: } The remote control software that is meant to run on an external PC for configuration purposes
\end{itemize}

\begin{figure}[h]
	\centering
	\includegraphics[width=0.7\textwidth]{./design/SoftwareArchitecture}
	\caption{Overview of the interactions between the various software components and their communication.}
	\label{fig:SoftwareArchitecture}
\end{figure}

\subsection{Development Language Choice}
The software is written in \cpp\ for compatibility and performance reasons. All the device drivers provide libraries in either \clang\ or \cpp, while some drivers such as the library for the netHAT---called CIFX---are only provided in \clang. 

The topic of performance between languages and systems is one of heated debate, however \cpp\ was chosen for this project due to the ability to program comfortably in a higher-level language, while having the ability to \textit{\enquote{drop down}} into \clang. The \clang\ Programming Language is often the benchmark for higher-level programming languages when programming for Real-Time Systems due to its predictability and the ability to run operations with few layers of abstraction on memory directly \cite{pizlo2010}. 

Furthermore, since the scale of the processing unit of the program is relatively small, the benefits that come from using a higher-level programming language---such as increased productivity, organization, and re-usability \cite{pizlo2010}---are not strictly necessary.

As shown in \autoref{fig:processinglib}, the main functionality of the processing unit includes:

\begin{itemize}
\item Image Pre-Processing i.e.\ rotation, skew, cropping
\item Curve-Fitting
\item Cross-Correlation
\end{itemize}

\begin{figure}[h]
	\centering
    \includegraphics[width=0.8\textwidth]{./design/ProcessingLibrary}
    \caption{Representation of the how the processing unit is called by different components of the program.}
    \label{fig:processinglib}
\end{figure}

These are implemented in \clang\ as much as possible. This is then encapsulated by a \cpp\ wrapper. This provides ease-of-use on the remote side, where processing is not real-time critical, while still allowing the local side to directly call the \clang\ processing functions.

\subsection{FlowRemote -- Remote Control GUI}

FlowRemote is designed in order to allow for easier configuration and calibration of the setup, as well as enabling the engineer to do so remotely. The idea being that---once the Raspberry Pi and LIDAR sensor have been installed over a conveyor system and a network connection---the engineer no longer requires a direct physical connection to the Raspberry Pi in order to configure and calibrate the system. \autoref{fig:flowremotegui} shows the design of the GUI.

As described in \autoref{fig:flowremote}, FlowRemote allows the engineer to remotely preview the raw sensor data, run pre-processing on it, configure the processing parameters and deliver those back to the local processor running on the Raspberry Pi. 

\begin{figure}[h]
	\centering
	\includegraphics[width=0.9\textwidth]{./design/FlowRemote}
	\caption{Overview of the FlowRemote features and configurable parameters.}
	\label{fig:flowremote}
\end{figure}

\begin{figure}[H]
	\centering
	\includegraphics[width=\textwidth]{./flowremote}
	\caption{Design of the FlowRemote GUI.}
	\label{fig:flowremotegui}
\end{figure}

\section{Housing}
For the purposes of field-testing the project, a rudimentary housing was designed in CAD---see \autoref{fig:housing}---and 3D printed. The housing provided a small amount of protection from the environment for the otherwise bare Raspberry Pi.

\begin{figure}[h]
\centering
\includegraphics[width=\textwidth]{housing}
\caption{Isometric view of the underside (left) and topside (right) of the prototype housing.}
\label{fig:housing}
\end{figure}

The housing was constructed around the standard Raspberry Pi 4 Model B with the netHAT modules attached, allowing for the extra ports to be accessible through the housing as well.

This housing is naturally unsuitable for production use, as it lacks adequate weather and shock-proofing.