\chapter{Validation}\label{chap:validation}

As was briefly touched upon in \autoref{sec:developmentphases}, the development of this project was carried out in iterative phases. After each iteration, the developed features were validated for their functionality and suitability. This chapter will go over these stages---sandbox, laboratory and field-testing---in depth.

\section{Sandbox Stage}

The fundamental objective of the sandbox stage of development was to investigate the suitability of the Intel RealSense L515 LIDAR sensor.

In this stage, the testing was mainly to determine
\begin{enumerate}
\item if a connection to the sensor can be established
\item if the data received can be visually represented and manipulated---see \autoref{fig:mug}
\item if the accuracy of the data was within a tolerable range
\end{enumerate}

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{prototype/mug}
\caption{Visual representation of the sensor data plotted as a color-graph.}
\label{fig:mug}
\end{figure}

In order to accomplish this, a precursor to the FlowDAR software was developed that was connected to the sensor directly over USB. The aim of the software was to process the raw sensor data in order to determine the cross-sectional area of an object upon a flat plane. In this case, a small cardboard box was placed against a wall.

As represented in \autoref{fig:prototype_program}, the software first calibrates itself to the flat plane---the wall---using linear regression to generate a straight-line. Then upon placing the object on the plane, using the techniques discussed in \autoref{chap:design}, the cross-sectional area of the object could be measured.

An object with the cross-sectional area of \SI{3150}{\milli\meter\squared} was used in this validation. The software measured \SI{3288}{\milli\meter\squared}, yielding an error of \SI{4}{\percent}.

\begin{figure}[h]
  \centering

  \begin{subfigure}[t]{0.45\textwidth}
    \includegraphics[width=\textwidth]{prototype/fullrange.png}
    \caption{Full scanning range of the LIDAR sensor, recording the geometry of
      a wall and bookshelf.}
    \label{fig:fullrange}
  \end{subfigure}
  \hfill
  \begin{subfigure}[t]{0.45\textwidth}
    \includegraphics[width=\textwidth]{prototype/wallpreview}
    \caption{A zoomed in selection of just the straight section of wall.}
    \label{fig:wallpreview}
  \end{subfigure}

  \medskip

  \begin{subfigure}[t]{0.45\textwidth}
    \includegraphics[width=\textwidth]{prototype/linearregress}
    \caption{The selection is then used to run a regression algorithm, to
      produce a straight line that should represent the wall's surface.}
    \label{fig:linearregress}
  \end{subfigure}
  \hfill
  \begin{subfigure}[t]{0.45\textwidth}
    \includegraphics[width=\textwidth]{prototype/calib}
    \caption{Calibrating the sensor using an object of known length.}
    \label{fig:calib}
  \end{subfigure}
  \medskip
  \caption{Operation of the prototype software.}
  \label{fig:prototype_program}
\end{figure}

\section{Laboratory Stage}
The development of the fundamental features of this project was done in an iterative process of rapid prototyping and testing. In order to accomplish this, a controlled environment that can be easily accessed and modified must be established. This setup was realized in the Telelaboratory\footnote{Laboratory for the development of remote systems at the Faculty of Electrical Engineering, University of Applied Sciences Düsseldorf}.

The setup consisted of a miniature looped conveyor belt system. The looped nature of this conveyor system was advantageous, as it could be loaded with material that would continuously circulate. This allowed development to be carry on uninterrupted and even remotely if necessary. The objects used to simulate material on the belt were miniature cars that were chosen simply for their availability and simple geometry.

The LIDAR sensor was mounted using its ISO 1222 tripod mounting point on a regular camera tripod and positioned over the conveyor belt. The Raspberry Pi was connected to the laboratory network which allowed for configuration and testing to be done over the network. Using a VPN tunnel, further configuration and testing could also be done remotely from outside the laboratory network.

\subsection{Cross-Sectional Area}
The development and testing of operations to measure Cross-Sectional Area were straightforward as described in \autoref{chap:design}.

\begin{table}[h]
\centering
\begin{tabular}{| c | c |}
\hline
Object Cross-Sectional Area & Uncertainty \\ \hline
\SI{20}{\milli\meter\squared} & \SI{10}{\percent} \\ 
\SI{120}{\milli\meter\squared} & \SI{4}{\percent} \\ \hline
\end{tabular}
\caption{Uncertainty of Cross-Sectional Area measurement for different sized objects.}\label{table:cross_uncertainty}
\end{table}

As shown in \autoref{table:cross_uncertainty}, measurements of the miniature cars---with cross-sectional areas of \SI{20}{\milli\meter\squared}---had a relatively high uncertainty of around \SI{10}{\percent}. The uncertainty was reduced to \SI{4}{\percent} when using a cardboard box of a larger size. This however, is to be expected according to the specified uncertainty of the LIDAR sensor at \SI{1}{\meter}.

\subsection{Belt Velocity}
In order to determine the accuracy and reliability of the cross-correlation algorithms, they were compared to the known velocity of the belt. This was determined simply by recording the time taken for an object to travel a known length of the belt and was determined to be \SI{11}{\centi\meter\per\second}.

While the algorithm did return a value of \SI[separate-uncertainty = true]{11(1)}{\centi\meter\per\second} while the object was in the center of the area-of-interest, the error varied up \SI{100}{\percent} while the object was near the edges of the area-of-interest. This rendered the results of the tests inconclusive. One reason could be due to the small size of the objects---height up to \SI{5}{\milli\meter}---causing strong deviations near the edges, where the sensor uncertainty is usually at its highest.

Since this error may be caused simply by the small-scale laboratory conditions, the algorithm was brought forward to be tested in field-conditions with little change.

\subsection{PLC Test}
Once the development of the Profinet interface was complete, a simple test project involving a PLC was designed in order to test the interface's functionality. The existing setup was connected via the netHAT's Ethernet interface to a PhoenixContact PLCNext controller. The PLCNext controller was wired to an I/O kit that provided various actuators and LED outputs for prototyping purposes.

The values sent to the controller by the FlowPi over the Profinet interface were the following:
\begin{itemize}
\item Cross-Sectional Area
\item Band Velocity
\item Volume Flow
\end{itemize}

A simple PLC program was written to activate an output---in this case, turning on an LED---whenever the Cross-Sectional Area was over a certain threshold value.

The FlowPi software as well as the Profinet interface were shown to be functioning as the LEDs lit up in a robust manner whenever a miniature car passed under the scanning area of the LIDAR sensor. A rigorous measurement of the latency was not carried out, however the latency was deemed to be under one second.

\subsection{Linux RT-Patch}
In order to test the effect of the Linux RT-Patch, a simple test comparing the jitter values of Profinet-IO communications was conducted.

The system was connected over Profinet to a Virtual PLC running on Codesys, and the maximum jitter values as reported by Codesys were recorded. During the \SI{30}{\second} duration of the test, the cross-section area was processed and delivered.

The results show that the RT-Patched kernel had a maximum jitter of \SI{2166}{\micro\second}, which was \SI{26}{\percent} lower than the normal kernel. This lower jitter may be indicative of higher-determinism of the system.

\section{Field-Testing Stage}
The field-testing stage was carried out at the Siep Gravel Quarry\footnote{Siep Kieswerk GmbH \& Co. KG in Jülich. See \nameref{chap:ack}.}---see \autoref{fig:photo_overview}. There, a bucket loader was being used to excavate gravel into a hopper. The hopper first filtered out larger rocks and boulders through a set of evenly spaced rods. Acting as a buffer, the hopper would continuously load a conveyor belt with gravel.

The quarry currently uses a conventional belt scale system that was placed under the conveyor belt. This simultaneously measured both belt velocity and the material mass flow---in tonnes per hour---delivering the values to a PLC in a nearby control box. 

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{photographs/overview}
\caption{Siep Kieswerk GmbH \& Co. KG in Jülich where the field testing was carried out.}
\label{fig:photo_overview}
\end{figure}

\subsection{Setup and Testing}
The LIDAR sensor was attached to a walkway that went over the conveyor belt---see \autoref{fig:beltview}. The sensor must be placed at a minimum distance of \SI{0.5}{\meter} from the belt, in addition to clearance accounting for the height of the gravel on the belt as well. In this case, the sensor was placed at a height of \SI{1}{\meter} from the belt. The housed Raspberry Pi and various connections were also attached to the walkway.

\begin{figure}[h]
\centering
\includegraphics[width=0.75\textwidth]{photographs/beltview}
\caption{The view of the conveyor belt and material as seen by the sensor.}
\label{fig:beltview}
\end{figure}

A standard home-grade wireless access point was used to provide a local network, through which the configuration of the system could take place.

Once all the devices were connected and turned on, the pre-configured Raspberry Pi connected itself to the wireless access point that was reachable by the engineering laptop. A connection between the FlowRemote configuration software and the FlowPi processing software could be established and configuration could begin.

Configuration took place in the regular manner that was refined during the laboratory testing.

\subsection{Issues}
Upon the establishment of the connection and the preview of the sensor's data, it was immediately apparent that parts of the sensor image that contained the distance of the conveyor belt was zero. Gravel and other objects placed on the belt had regular and non-zero depth values. This meant that the conveyor belt not being ``seen'' by the sensor. In other words, the reflectivity of the conveyor belt near the wavelength of the sensor---\SI{860}{\nano\meter}---was too low to consistently and accurately calculate depth.

\subsection{Solutions}
This issue meant that the fundamental principles in which the operation of the measurement software depended on, was unusable. A new approach must be investigated by finding a method to obtain the geometry of the conveyor belt, and using a modified algorithm to calculate its volume without continuous access to the conveyor belt's geometry. In essence, the gravel will appear to be ``floating'' in the sensor image. Such an algorithm may use more precise measurements of the conveyor belt during the configuration and calibration phase, in order to make estimates of the conveyor belt geometry during live processing.

Ultimately however, the implementation of a solution to this issue requires another iteration of development that is not within the scope of this project.