ICACT20230138 Slide.15
[Big slide for presentation]
|
Chrome Text-to-Speach Click!! |
 |
Thanks for watching!
|
|
ICACT20230138 Slide.14
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This is our references articals. Our previous research is the reference number 14 and 16.
|
|
ICACT20230138 Slide.13
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
and, If we use a wide-angle lens with a wide field of view of the camera, we can monitor a wide area, but this may cause a problem that the image is relatively severely distorted. In addition to visual problems, this image distortion is especially problematic when physical coordinate numerical calculations such as exact position calculation through image analysis are required. Obtaining the distortion-corrected image coordinates from the distorted image coordinates is a very difficult problem without a closed form solution, and the development of technology to correct images in real-time moving unmanned aerial vehicles such as helikites is a problem to be solved.
The operation method of image processing mission equipment can be largely divided into three methods as shown in figure of bottom of slide. The first method is to install an image acquisition and processing device on an aerial device such as a helikite, and obtain image processing results directly from the air device. The second method is to use a centralized image processing center such as a cloud center. And the last method is to operate the cloud at the end of the network, similar to the operation in MEC (Multi-access Edge Computing) of the 5G network. In the case of port monitoring, since it is difficult to build a network infrastructure, additional research is needed on which method is suitable for port monitoring.
|
|
ICACT20230138 Slide.12
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
We propose AI based harbor surveillance system consisting of object recognition and anomaly situation determination. In the object recognition, YOLOv5, which had the best performance with an mAP of 0.953, was used. The abnormal situation was determined through the distance between the recognized objects. The comparison of the determination results from real and estimated distance are same.
and We have several limitation for this research.
First limitation is lower recognition rate for small object such as person and bike. It is because that, YOLO model determines
objects by dividing a fixed cell grid to match realtime performance. In order to solve the problem, we will research the object detection model for considering small size target scenario.
Moreover, there is another limitation in the early detection of anomaly situations. We set the criteria for anomaly situations based on only distance that is less than 20 meters. According to the simple criteria, in the experimental results, only 1.5 seconds was given from anomaly detection to collision. In future research, it is necessary to classify the warning stage so that anomalies can be recognized even at a longer distance based on the direction and speed from AIS information.
|
|
ICACT20230138 Slide.11
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The initial distance between the two ships is about 330m, and the simulation was conducted so that the distance between the two ships decreased as the experiment time passed. The actual distance between the ships was calculated from the latitude and longitude of the two ships according to the time obtained from the simulator, and the estimated distance between the two ships, a recognized object, was derived based on the image. The criteria for determination an abnormal situation was classified into a dangerous state if the distance between two ships was less than 20 meters, and a safe state otherwise. Left table shows the results of the abnormal situation judgment experiment from about 3 seconds before the collision. The total length of the experiment is about 21.5 seconds. Although the average distance error between the real distance and the estimated distance is 3.4m, the determination results achieved a high success rate in the experiment environment. Right Figure show two examples of video outputs for safe and warning cases.
|
|
ICACT20230138 Slide.10
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
An image data from the camera is two-dimensional pixel data. It is not easy to find out the distance between ships using only pixel coordinates. However, the actual ship is located at a point in the three-dimensional Cartesian coordinate system. If the three-dimensional position of the ship can be determined by transforming the pixel coordinates, the distance between the ships can be calculated. For obtaining 3D coordinates using a single camera, we make two assumptions that all objects are at sea(or ground) level and that the angle of one pixel is the same. The conversion process is schematically illustrated as in figure.
|
|
ICACT20230138 Slide.09
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
We consider an approach situation that can cause a ship collision as an abnormal situation and establish a system that can judge it. Since it is quite difficult to obtain the ship collision situation data in the real offshore, the situation was simulated using the ship simulator (the simulator model name is K-Sim Navigation, and the manufacture and configure company is Kongsberg Digital). In the ship simulator, latitude and longitude and simulation image data of two ships were acquired at three samples per second under the condition that the camera’s altitude and vertical angle were fixed.
|
|
ICACT20230138 Slide.08
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The results of calculating Precision, Recall, and mAP for each of the three models are shown in Table. In the performance comparison, the object recognition rates are 0.844, 0.715, and 0.953 in SSD, YOLOv3, and YOLOv5, respectively. Therefore, in the next section, YOLOv5, which showed the highest performance, was used.
|
|
ICACT20230138 Slide.07
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
A overall process for Harbor Object Detection shown in Figure.
|
|
ICACT20230138 Slide.06
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
For object recognition for harbor monitoring, SSD, YOLOv3 and YOLOv5 models were trained and compared performance using datasets, respectively. Since deep learning requires a large amount of learning data, data augmentation techniques such as color and brightness adjustment randomly and random perspective were used to overcome the small amount of learning data. To reduce the likelihood of a particular result for a particular training data, we use k-fold validation to separate the training and test data for each experimental (i.e., k=5 in this study).
|
|
ICACT20230138 Slide.05
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
In order to learn the object recognition algorithm, we created an image dataset for training. Ships, vehicles, and people were found in the harbor images acquired through helikite and labeled using Yolo mark. The number of images used is 11,318, and the number of objects in the images is 64,151. table shows the number of objects by object type. Also, a sample of acquired images is shown in right figure.
|
|
ICACT20230138 Slide.04
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
This slide shows an overview of a video-based harbor surveillance system.
The conventional system that could continuously acquire images using helikite was configured to automate monitoring
of the port using AI. Harbor surveillance automation consists of two main steps. In the first step, it recognises
objects such as ships, vehicles, and people from helikite image data. In the second step, the recognized object is used as input
data to determinate an abnormal situation that detects the risk of collision between ships.
|
|
ICACT20230138 Slide.03
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
A UAV has been developed and commercialized in various forms. In the types of UAVs, there are rotary-wing and fixed-wing, and recently, there is also a form of an unmanned airship. Quadcopters are the most widely studied and used rotary wing type because they can take off and land vertically and can be manufactured in a compact size. The unmanned airship type has the advantage of being able to mount heavier weights and extend operating time. Helikite is a hybrid system of airship and balloons. Helikite does not have a power source like an airship, but and has additional lift and weather stability due to the shape of a balloon attached to a kite wing. So the helikite can be operated for a long time and stable image acquisition is possible, but manpower is continuously required to monitor the image. So, We purpose the surveillance system based on AI algorithm.
|
|
ICACT20230138 Slide.02
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
When ships of various types and sizes enter and depart from ports, the risk of collision between ships is high. In particular, collisions between large ships can lead to major accidents. In order to avoid crashes, it is important to quickly identify hazardous situations in advance. For this purpose, video surveillance, radar, automatic identification system (AIS), CCTV, etc. are being used. However, there is a limit to seeing the entire port as CCTV is fixedly installed. Monitoring using drones is also difficult due to limited batteries. As a way to overcome this, the use of An unmanned aerial vehicle (UAV) such as helikite has been proposed.
|
|
ICACT20230138 Slide.01
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
The structure of this paper is presented in three parts, firstly a background introduction, thirdly the system design of AI Based Harbor Surveillance System and finally a summary of the whole paper.
|
|
ICACT20230138 Slide.00
[Big slide for presentation]
[YouTube] |
Chrome Text-to-Speach Click!! |
 |
Hello, My name is Dongkoo Shon. I am researcher from the Electronics and Telecommunications Research Institute in South Korea. Today, I will brief you on my research subject on A Study of AI-Based Harbor Surveillance System.
|
