A Novel Algorithm of Distance Calculation Based on Grid-Edge-Depth-Map and Gyroscope for Visually-Impaired

This paper presented a new algorithm for determining the distance of an object in front of a stereo camera placed on a helmet. By using a stereo camera with a Sum of Absolute Difference with a Sobel edge detector, our previous Grid-Edge-Depth map algorithm could calculate the objects’ distance up to 500 cm. The problem started when a vision disability person used the device with an unﬁxed stereo camera angle. The unspeciﬁed angle caused by the helmet’s movement inﬂuenced the distance calculation result. This novel process started with calculating the distance from a Grid-Edge-Depth map considering unﬁxed angle data of the x-axis from a gyroscope sensor placed on the stereo camera using the trigonometry formula. The angle data used was the x-axis data. The distance measurement results by the system were then computed based on the unﬁxed angle compared to the actual distance. The test was carried out with three scenarios which required the user to stand at a distance of 100 cm, 125 cm, and 150 cm from a table, chair, or wall, with 30 tests for each scenario. The test results showed an average accuracy of 96.05% with three experimental scenarios, which meant that this machine was feasible to implement.


INTRODUCTION
As God's most perfect creatures, humans can do various things daily. However, some are given limitations or cannot do so in certain respects. There are various types of disabilities or disabilities. One is visual impairment [1][2][3][4][5]. A person with disabilities in seeing is usually gifted with other abilities, such as hearing, that are better than ordinary people who are expected [3,5,6]. Visual disabilities use a navigational aid as a stick when walking or doing activities in specific environments when no one accompanies them. An ultrasonic stick is one of the innovations with limited functionality [1][2][3][4]. Unfortunately, there are not many assistive devices that people with visual disabilities can use without having to perform eye transplant surgery.
Meanwhile, there are not many visual aids that can be used directly without involving medical action at all [7]. In other words, there must be an assistive device that people with disabilities need to wear and be given a brief lesson on how to operate the tool, and then the viewing aid is ready to use [6,8,9]. In avoiding objects when navigating in a specific environment, the problem is how an aid device can accurately measure a distance [10,7,11]. Previous research was conducted to measure the distance using a stereo camera. The algorithm is called Grid Edge Depth map (GED-map). GED-map can measure an object's distance using a stereo camera built using two 720-pixel web cameras and can measure objects up to 500 cm. Initially, this method was developed for wheeled robots with a holonomic drive system to avoid obstacles when navigating in specific environments [12,13].
This research has developed a prototype for people with visual disabilities by utilizing the results of video acquisition using a stereo camera [14]. It was implemented in a helmet similar to that used on wheeled robots [12]. The camera is mounted on an ordinary cyclist's helmet and is processed using a portable computer inserted into a backpack. The problem with helmets equipped with stereo cameras is that the camera's viewing angle has not been considered [12]. The unfixed angle of the stereo camera will affect the distance between the user and the object or obstacle in front of him. Next, the user's anticipation of the presence of this obstructing object may also be too late or even wrong. Then a warning by the system may become a false alarm. Preliminary testing of the device's computational process has been done by setting a fixed camera angle to determine the accurate distance to objects or objects visible to the camera. The test results show that as long as the camera angle does not change, the accuracy of the computational results of actual distance measurements will remain above 90 percent [15,16]. The computational processes on the actual distance of objects in front of the user must be carried out based on the acquisition results of stereo cameras with relatively variable angles [11]. It could be that the camera's angle mounted on the helmet will be bigger or smaller. Of course, when used, the device user is not always at a fixed angle which is used as a constant in computing by the system. This research is a continuation of previous research that tested. Algorithm testing has been carried out using a wheeled robot with a holonomic-drive system model. The test results show that this algorithm can avoid obstacles in the experimental environment with additional rules embedded in the robot.
Algorithm testing on wheeled robots will differ from testing involving humans as users [17]. Because it is a digital program, what is obtained based on measuring the distance of objects using the GED-map algorithm can immediately be used as the basis for the following command by the computer to the robot, namely in the form of direct commands for the robot's motion. Information in the form of temporary sounds and vibrations can allegedly be used to guide users of this visual aid in moving or navigating the surrounding environment. Unlike the case of providing information to humans, especially people with visual disabilities, it is necessary to find suitable formulas and indicators so that humans can utilize this distance information as an assistant in the context of their navigation. Of course, the prototype of this tool must be carried out and used efficiently. The results of the research are expected to become the forerunners of portable and integrated viewing aids that use video processing that is more compact, lighter, and more energy efficient. Moreover, the research results are expected to be a reference for further research.
The novel algorithm is intended for determining the distance of an object in front of a stereo camera. The distance variable is combined with the triangle formulas using axis data from a gyroscope sensor. Moreover, all the stereo cameras and the gyroscope have been placed on a helmet. This article presents a few parts of the research processes, and the proposed algorithm will be explained briefly in the next chapter. Finally, the last chapter will present the result of the research, some unique findings, and the research conclusion.

RESEARCH METHOD
Generally, our proposed method of measuring the object's relative distance consists of two parts. The first part is building the GED map. The second part was the novel algorithm of this study. This second process was reading the variable x-axis from the gyroscope and calculating the actual distance between the user (visually impaired person) using the triangle formula.

Hardware Requirements
In order to get the experiment data, this study used some hardware. First, the cyclist helmet is assembled with a stereo camera. The stereo camera position is parallel to the top of the helmet. In this case, the 720p webcam is used. Figure 1 depicts how the stereo camera installed on the helmet combined with the gyroscope sensor. A gyroscope sensor MPU-6050 was used. The module provided a 3-axis variable (x, y, and z-axis), and the x-axis was used in this study. Besides, the module is relatively cheap and easy to get in Indonesia's online marketplace. Figure 2 illustrates how the visual aids are worn by a person who acts as a visually impaired. A cycling helmet was equipped with a stereo camera and gyroscope sensor module. Whereas the laptop was carried (using a backpack) by the model of a visually impaired person as the following hardware was used in this study. This laptop is the central processor of all the processes that will be present in the following sub-section. The laptop's specification was a core i7 processor, 16 GB of RAM, and 2 GB of dedicated VGA memory. The connection between the laptop and the stereo camera was using the USB port. Besides, the gyroscope module used the Arduino Mega mini with the ATMega2560 as an intermediary module to the laptop via a USB port. Three USB Port was used totally in this study.

Grid-Edge-Depth Map Building
The novel process started with calculating the distance from a GED map building, as shown in Figure 4. The stereo images were acquired using a calibrated stereo web camera with 320x240 pixel resolution, and then the histogram equalization was applied. Then the images were converted into grayscale to save memory demand. Therefore, using the camera calibration parameter, image rectifications were applied. In order to obtain a disparity map, the stereo matching algorithm called the Sum of Absolute Difference (SAD) is used with the block comparison technique [8,9]. The system subtracts each pixel in the template (left image) from the corresponding pixel in the block on the right image. Then the absolute value of the differences could be calculated. These differences are then summarized, and a single value is derived that roughly measures the similarity between the two image patches. A lower value means a block pixel from the left and right images is close to being similar. The GED map size of the block is 25 pixels [10]. The depth map was built based on a disparity map by calculating the camera baseline and focal length [13,18,19]. Then it was divided by x for every pixel of the disparity map. The stereo camera baseline and focal length are 7 cm and 428 pixels, respectively.
In order to calculate a specific object distance, obstacle edge detection using the right image is applied with the Sobel edge detection algorithm [6]. The right image is a comparator image in building a disparity map, so this right image is used in this process. This edge-detection process is resulting a binary image [15]. The white color of the edge detected obstacle value would be one, and the others will be zero (the color is black) [6].
Furthermore, substituting the edge detected object's pixel with depth value at the same pixel position. Moreover, every zero pixels will be replaced with a value of 500 as the farthest distance measuring ability of the system. This process is resulting an edge-depth image. The last step in building the GED map is dividing the edge-depth map into five rows and five columns (25 grids). The row and the column size for each grid were 48 and 64 pixels, respectively, since the size of the edge depth map was 320x240 pixels [12]. Grid 3-3 calculates the accurate distance between the user and the object. In the GED map will be 25 distance values for each grid. Furthermore, the system will compare all the depth values of a grid and decide the lowest value as the distance of the detected object of the grid. Finally, the rest of the proposed method is depicted in Figure 5.  Figure 2 describes the distance between the stereo camera and the object (Sdist), and of course, should be more processes to obtain the accurate distance. Figure 5 depicts the flow chart of the proposed algorithm, including the GED map-building process, as explained in sub-section 2.2. The following process reads data x-axis from the gyroscope module using microcontroller ATMega2560 (Arduino Mega mini). Then the system will pick the data up from the GED map. It was distance data from one of the grids. Grid 3-3 was chosen because it was the center distance data from all 25 grids. It will be the Sdist variable.

Proposed Method
The following process reads the x-axis data from the gyroscope sensor or the camera axis (Ca) (B-axis in Figure 3). Since the measured distance from the stereo camera, called the GED map, becomes a diagonal distance (or the hypotenuse in Figure 3), the system needs to recalculate the horizontal distance between the people who uses this device toward the obstacle in front of the user (UoD or User to object distance or the adjacent). According to Formula (1), the adjacent could be calculated using Formula (2) or Formula (3). All the experiments were divided into three scenarios with 100 cm, 125 cm, and 150 cm objects, respectively, as the manually measured distance to compare with the calculated. Previously, the difference between the user-to-object distance (difC) and the manually measured distance (Mdist) was calculated using Formula (4). Furthermore, the experiment calculates the accuracy of each measurement (AcM) in all the scenarios using the Formula (5). Finally, the average accuracy of each scenario's experiment is calculated using Formula (6), and 30 measurements for each scenario are enforced in this research.
Avg AcM = ΣAcM total measurement (6) Figure 5. The Flow Chart of the Proposed Method Table 1 shows the comparison between the previous work and the proposed study. All the previous work and the proposed study used the stereo camera. They calculated the relative distance between the camera and the object in front of it using their proposed algorithm or method. ISSN: 2476-9843

RESULT AND ANALYSIS
The proposed method depicted in Figure 4 and Figure 5 results in some of the following images, and Figure 6 is one of them in the GED map-building proses. The initiation process was acquiring images from the stereo camera resulting in an unbalanced histogram that needed equalization. Figure 6 depicts the basic stereo images, a pair of 320x240 pixels, and their histogram. The previous study in [7] used 672 x 376 pixels of a stereo image. Then the histogram equalization process as a preprocessing is conducted while edging detection. The system uses the right image in the edge detection process.
Furthermore, the disparity map building uses the sum of absolute difference algorithm. Then the depth images are built based on the disparity images. The result of this process shows in Figure 7.  The depth map image is then substituted in the subsequent process into an edge image of the object. First, a new image is obtained as an object edge feature image with a value of object distance or depth at each pixel. The image is then named edge-depth map. Then, the edge-depth map is divided into five sections, horizontally and five vertically, to determine the position of objects horizontally and vertically. These sections are then referred to as the grid. Finally, each grid is calculated for the minimum value to bring up a single value attached to each grid. One of the outlines of this research is to design a distance measurement model and an estimate of the position of objects seen by stereo cameras mounted on a helmet. This model is called a Grid-Edge-Depth map and abbreviated as a GED map. The Grid-Edge-Depth map is shown in Figure 8 The helmet is also equipped with a gyroscope module to deliberate the axis variable of the camera. This process is to obtain information on the relative position of the object/obstacle to the stereo camera. The proposed idea is to divide the edge-depth map image into several grids, both horizontally and vertically. It is based on previous research. Moreover, the grid size must be considered to improve accuracy in measuring objects' distance and relative position. So for the decision-making process of motion direction, we need a grid in the middle of the edge-depth map image as the middle position of the robot's vision through a stereo camera. If the edge-depth map image is divided into three, this number is still too small for later decision-making. It has therefore chosen to divide the edge-depth map image into five grids horizontally. Vertically. The edge-depth map image is also divided into five grids. The entire grid in the edge-depth map image is 25 grids. The result of this process is called a grid-edge-depth map. The GED map contains information on the distance of the object/obstacle. It also gives information about the relative position of the object/obstacle to the camera. Each grid will have 64 columns and 48 rows. It was obtained from an image size of 320x240 pixels divided by five, each horizontally and vertically. The proposed method's experiment was conducted and has been shown in Table 2, Table 3, and Table 4, respectively. In the first scenario, we test the proposed method by calculating the distance of the object 100 cm in front of the user. In this first scenario, all the calculated distance from the proposed method is more significant than the manual distance measurement. The experiment result shows that the average accuracy of the whole 30 times calculation was 96.31%, with the most significant difference between the manual distance measurement being 10.59 cm and the minimum calculation difference being 2.27 cm. The second scenario resulted in the average calculation accuracy between the proposed method and the manual distance measurement (in this case, 125 cm), which is 96.10% from 30 times the calculation trial. Furthermore, in this second scenario, all the calculated distances from the proposed method are also more prominent than the manual distance measurement, with the most significant difference between the manual distance measurement being 7.27 cm and the minimum calculation difference being 4.19 cm. The average calculation accuracy between the proposed method and the manual distance measurement (in this case, 150 cm) obtained from the third scenario was 95.74% with 30 times the calculation trial. In this third scenario, all the calculated distances from the proposed method are lower than the manual distance measurement, with the most significant difference between the manual distance measurement being 7.10 cm and the minimum calculation difference being 3.95 cm. The first, second, and third experiments show that the change of the camera's x-axis obtained from the gyroscope significantly changed the calculated distance. The maximum x-axis of the stereo camera is 43.22 degrees, 42.55 degrees, and 41.35 degrees for the first, second, and third scenarios. Besides, the minimum x-axis of the stereo camera is 43.02 degrees, 42.20 degrees, and 40.85 degrees for the first, second, and third scenarios. Based on the triangle principle, if the camera axis is lower than before, the stereo distance will be increased, and the calculation distance (user to object or obstacle distance) will also be increased. The best average accuracy of all the experiments is obtained from the first scenario, with 96.31%.

CONCLUSION
The proposed algorithm considering the x-axis variable obtained from the gyroscope resulted in a significant result. The x-axis variable, as shown in the experiment result in table 2, table 3, and table 4, significantly influences the distance calculation based on the triangle formula. That was one of the contributions of the proposed algorithm. Besides, compared to previous studies in [7], there is an increase in the distance of objects that can be measured, that is, 64-500 cm, whereas their result was 40 cm to 300 cm. This study uses 320 x 240 pixels @11 fps of the stereo image, while [3] uses 1280 x 720 pixels @15 fps, [7] uses 672 x 376 pixels @15 fps, and in [20] uses 672 x 376 pixels @100 fps. In our proposed method, the time of image acquisition by the stereo camera is used at 11 fps in 0.09 to 0.14 seconds. The average accuracy of all the experiments is 96.05% which means that the proposed method is feasible to implement. This proposed method resulted in a real-time distance measurement. The subsequent research could find out ISSN: 2476-9843 how to minimize the size of the processor device, which used a slim laptop with a weight of 1.4 kg in this study.