Usefulness of semi-automatic tools for airborne minefield detection

Usefulness of semi-automatic tools for airborne minefield detectionP. Druyts, Y. Yvinec, M. AcheroySignal and Image Centre, Royal Military Academy, Belgiumhttp://www.sic.rma.ac.beAv. de la Renaissance, 301000 Brussels, BelgiumPascal.Druyts@elec.rma.ac.beYvinec@elec.rma.ac.beAcheroy@elec.rma.ac.beAbstract - Potentialities of airborne minefield detection are nowadays investigated. Such approaches are based on the detection of minefield indicators on mono or multi-sensor airborne images. Most of the indicators will have to be located in vast area using high-resolution imagery. This leads to a tremendous amount of data to be interpreted. Hence semi-automatic detection techniques must be used to pre-filter the input information flux or the user will be flooded by the incoming data. In this paper, we illustrate and evaluate such a semi-automatic technique for the detection of un-obscured anti-tank mines. The potentiality of semi-automatic tools for the detection of other mine indicators is also discussed.IntroductionLandmines are the cause of a huge and world-wide humanitarian disaster as about 110 million landmines are scattered in 64 countries around the globe. The currently used technology is not efficient. Therefore de-mining campaigns are slow and expensive. The major problems are linked to the high rate of false alarms and the poor localisation of mine polluted areas. Most often, the really polluted area is quite smaller than the suspected one and a lot of time is lost demining clear areas [2].Nowadays, this problem is receiving an increasing attention from the international community. Research projects are launched in several complementary directions such as:q Development of new sensors [8],q Adaptation and fusion of off-the-shelf sensors[1], [3],q Airborne detection of minefields [4], [10],q Mechanisation of scanning (vehicle or robot mounted) [5],q Mine destruction or neutralisation,q Mechanical clearing (roller, flail, etc.).A survey of current technology research can be found in [1].In this paper, we will discuss more in-depth the potentiality for airborne detection of minefields. More precisely, we will focus on the description of semi-automatic tools that are needed to support such detection.Need for airborne surveyThe delimitation of mine polluted areas has to be performed before the actual demining begins. This delimitation is performed during the mine surveys level 1 and level 2. In level 1 general survey, all locally available information such as hospital casualty reports are gathered to get a first idea of the location and extend of mine polluted areas. Then in level 2 technical survey, demining teams are sent to those suspected areas to reduce their size whenever possible and mark them.Currently, the minefield marking is quite imprecise. This is due to several reasons amongst which:q Imprecise, incomplete and sometimescontradictory information is gathered,q Difficulty to collect the locally available information (limited means of communicationsetc.),q Lack of efficient centralisation of all demining related information at the country or regionallevel,q Lack of precise maps.All this leads to a significant waste of time. Indeed, the suspected area is often far larger than the really mined area. Furthermore several independent organisations are often active simultaneously and it sometimes occurs that both demine the same region due to lack of communication, centralisation of the information or inaccurate maps.Obviously, airborne sensed information might be quite valuable in this context. Indeed, it is an independent and up to date source of information. It may help to obtain a more precise localisation of thepolluted areas. As a by-product, such data allows to update the existing old and inaccurate maps.It is unlikely that airborne survey might replace on-field detection in the near future. Indeed, even if some mines may be detected using airborne sensors, the probability of detection will most probably be far too small. However, airborne survey may be quite effective for minefield detection because it is not necessary to find all the mines and furthermore, other indirect minefield indicators may be used.Need for semi-automatic detection tools To demonstrate that airborne survey may efficiently be used for humanitarian demining support, one should show that:q Reliable mine indicators may be found in the used imageryq Those indicators may be found in a reasonable timeframe using reasonable resources.Nowadays, experienced human photo-interpreters clearly outperform any automatic system when one considers the interpretation of a reasonable amount of data especially when completeness of the analysis or the handling of exceptional cases is considered. Indeed, human operators can adapt very easily to a new situation or make the best use of any collateral information.However, the evolution in the imaging technologies leads to an increase of the resolution and the number of available spectral bands. All this leads to an exponential increase of the amount of available data on one hand but on the other hand, the number of trained image analysts remains fairly stable and will certainly not grow exponentially for economic reasons. Furthermore, human beings dislike routine work.Imagine that mines may be found on 1cm-resolution images and that an area of 10 km by 10 km has to be scanned. An operator would typically spend1 minute on each 1,000 x 1,000 sub-image. The total needed time may then be estimated to 695 days or 10 man-years.In practice, the needed time may be reduced by a hierarchical approach in which some regions may first be rejected on a coarser image using other criteria. However, the above figures clearly show that human interpreters may expect to be flooded by the amount of information to treat if they are not supported by efficient semi-automatic tools.Therefore, the most promising approach for problems where the new high-resolution imaging capabilities have to be used is a semi-automatic system in which the machine pre-filters the incoming flux of information presenting regions of interest to the user. We argue that this approach leads to the best synergy between the analyst and the machine by combining the huge computation power of modern computers with the outstanding interpretation capabilities of human beings.Commercial software packages, combining aerial image manipulation with the management of vector-style geographical information, do not include such tools. What these software packages excel in, however, is low-level image processing performed by their huge libraries: filtering the image in order to obtain a smoother noise-filtered image, making the image sharper, eliminating some unwanted blurring effect, detecting edges and so one. Although we agree that tools dedicated to pure image processing may be quite useful, we believe that more‘intelligent’ semi-automatic tools are also promising.ApplicationThe project ‘airborne minefield detection: pilot project’ led by ITC is an example of an application where semi-automatic tools can be useful. The aim is to study the possibility of detecting minefields by airborne surveys. The project is divided in two phases.In phase I, the sensors and the algorithms were tested in a simulated minefield in Leopoldsburg (Belgium). This minefield is composed of several smaller minefields. For one of them, named ‘minefield C’, everything about the mines laid, including their accurate positions and depths, is known. The sensors used were colour, colour infrared and thermal infrared cameras and X- and Y-band radar. The images where mines could be found are high-resolution (around 1cm) colour and colour infrared images. In what follows, only these two images will be considered. Phase I is now completed.In phase II, the most promising sensors will be used for airborne surveys of a mine-polluted country in Africa.Selected Mine indicatorsThe detectable indicators are function of the selected sensors (spectral band, resolution, etc.). Direct and indirect minefield indicators ranging from the mine itself to an old military camp may be considered.In the first phase of the project, the semi-automatic detection of un-obscured anti-tank mines was considered. Obviously, minefields composed of such mines we are amongst the easiest to detect and other indicators have to be investigated. For complex mineindicators, it will probably be necessary to make use of a search strategy and to feed the system with expert knowledge [9].The initial choice was motivated as follows:q Ideal choice for a semi-automatic approach: simple model and high resolution needed.q Promising mine indicators have to be defined by the photo-interpreters. This was only donerecently for a test minefield simulated inLeopoldsburg (Belgium). Furthermore, semi-automatic tools do not have the humancapability to adapt to new situations. It istherefore crucial to develop detectors that aredevoted to promising indicators for the targetedregion. Hence the selection of the mineindicators was deferred to the end of this yearwhen images of real minefields are available.Building a mine (indicator) modelAny tool dedicated to the detection of mine indicators encodes (either explicitly or implicitly) a model of the searched indicator. The used model should be selected with care. The expertise of photo-interpreters is very useful for this step especially when complex indicators are considered. Due to its simplicity, the model used here was developed without their help. Therefore, the available images were visually inspected to find mine characteristic features. Most of the images were geo-referenced, and put together in a virtual image. The positions of the known mines were encoded together with their attributes. All this allows for an easy navigation through regions of interest centred on the known mines. For a recorded mine of interest the user may request the available attributes (type, depth etc.) as well as the available image channels. It is then possible to go from an available channel to another or to display all channels simultaneously in a common projection.The used mine modelSome vegetation anomalies (e.g. missing vegetation) can be seen where mines are laid. Unfortunately, a huge amount of such regions exists and to reach a reasonable false alarm rate, complementary mine characteristics (specific patterns [7], signature in different spectral bands, etc.) have to be introduced in the model. Therefore, in a first approach the typical spectral response of an anti-tank mines lying on the ground was used. A typical example of such a mine may be found in Figure 1. The image represents the blue band of a colour infrared image.The mine is the white area at the centre of the image (the blue component on colour infrared image is actually the response of the object in the green partof the spectrum).Figure 1 Mine used to tune the algorithmsAlgorithms for mine detectionThe algorithm is based on a maximum detection followed by a region growing in the blue band. The growing algorithm uses a rough model of the mine to compute a local contrast used to derive the growingstop criteria.Figure 2: Result of growing around a mine onCIR imageFigure 2 presents the results of the growingalgorithm. Figure 3 presents the blue band (on which the growing is performed) of the same area. As can be seen on the window displayed by clicking the mine location (see Figure 2), the type of the mine was encoded; it is an anti-tank mine lying on the ground. The mine is well detected but some other regions are also found. To reduce the false alarm rate, geometrical and radiometric attributes (using all available spectral bands) are computed for each mine candidate and a criterion based on those attributes is then used. Figure 4 presents the attributes used and the result after filtering. In the presented region all false alarms were rejected, keeping only the actual mine. Figure 5 presents the mine candidates found using the same approach on a part of the visibleimage.Figure 3: Blue band corresponding to Figure 2Figure 4: Attribute used and results after filteringFigure 5 Mine-like objects detected on visible imageDetection of partially occluded circlesAs explained above, the shape of the candidateregions may be used as discriminant feature. Mines lying on the ground often present circular shapes.However, the mine is often partially occluded and we developed an algorithm that is able to recognise circular shapes even if they are partially occluded.This algorithm is based on robust estimationmethods [6]. A measure of the circularity is returned together with an estimation of the radius. The radius provides a better estimation of the size of the object than the visible area (partial occlusion) and is thus more discriminating.The algorithm estimates a circle that goes through most of the pixels of a given list. This is useful when it is known that the pixels should lie along a part of a circle but some of them can be wrong and very far from the circle.Several triplets of pixels are selected in the list and the circles defined by these three pixels arecomputed. For each circle, the distances of all thepixels of the lists to this circle and the percentiles of these distances are computed. The circle selected is the one giving the smallest percentiles of errors. The number of triplets is computed to have at least one triplet without erroneous pixel, and thus a correct circle with a probability of 99 % taking into account the expected rate of erroneous pixels.Figure 6 shows a close-up of an anti-tank mine in colour infrared. Only a part of the mine is visible.The blue colour is due to the fact that the colourgreen is seen blue in colour infrared.Figure 6 Initial imageFigure 7 shows the contour of the mine detected.Because of the occluded part, the shape is notcircular.Figure 7 Contour of the mineFigure 8 shows the circle estimated from theprevious contour. It is a good approximation of the position and size of the mine. If an absolute co-ordinate system is available, it is possible to estimate the real size of the mine, which is an indication forthe type of mine.Figure 8 Circle around the mineEvaluationThe following images present the result of the automatic mine detection on the full visible and colour infrared images. Note that the size of thevisible image is about 400 MB and this image covers a region of about 500 square metres. The full image was processed in about 30 minutes on a Pentium Pro (200 MHz, 64 MB RAM) running under Linux. In Figure 9 and Figure 10, the position of known mines has been superimposed on both images (dots are known mines and crosses are mines proposed by the algorithm).On the visible image, a V-shaped minefield has been detected. Note that no mine of minefield C has been found. The anti-tank Mine 281 (lying on the ground and well visible) was first detected but rejected by the attribute based filter. Using a better filter criterion, it could be possible to keep that mine without increasing the false alarm rate but little effort was spent for this fine-tuning because we believe that an algorithm that learns from example should be used in practice. This could not be tested until now because the database contains too few examples of known and detected mines. In this context a learning scheme would probably lead to poor results (over-training).On the colour infrared image that presentsminefield C at a higher resolution but with only a small neighbouring region (the V-shaped minefield is not in the imaged region), three anti-tank mines,on and below the surface of minefield C, have been detected. Note that for the buried anti-tank mines, it is a stick lying near the mine that is detected. Even if the stick may be considered as a valid mine indicator, it was found ‘by chance’ since thealgorithm was not developed to find such objects. We concentrated our efforts on the detection of indicators that could be helpful in real situations. If it appears that such sticks are good indicators, a better detector will be developed.ConclusionA tool was developed to find anti-tank mine lying on the ground. Most of those mines were detected and the false alarm rate is reasonable. Even two buried mines were found because sticks left in the vicinity of the mines were detected. We believe that the false alarm rate could be reduced in the near future by some simple improvements. As an example many false alarms in the colour infrared are found in the trees. A tree detection algorithm could reduce the false alarm rate significantly.The human interpreters outperformed the semi-automatic system by their ability to integrate in their reasoning a great number of mine indicators: seven minefields out of nine were successfully detected with three false alarms. The most used and promising indicator was the alignment of soil perturbation.We admit that if image processing is only able to detect anti-tank lying on the ground, its usefulness in real situations would be quite limited. However, photo-interpreters are able to detect more minefields using other indicators. During phase II, the most promising indicators will be selected in collaboration with the photo-interpreters. Semi-automatic tools will then be developed to detect such indicators. Although limited, the results of the first evaluation presented in this paper have shown that significant process acceleration can be reached by means of image processing. By removing the false alarms found in the trees of the colour infrared image, the alarms found could be grouped in about 10 regions of interest. If the photo-interpreter takes 30 seconds to look at each region, 5 minutes would be needed to analyse the scene. When carrying out a full visual inspection at full resolution, the photo-interpreter would typically spend at least one minute for a region of 1,000 by 1,000 pixels. 150 such regions have to be analysed for a full coverage of the colour infrared image leading to about 2 hours of interpretation. A process acceleration of about 60 may thus be expected. Such a speed-up would be very useful in an operational context where the amount of data to be analysed would be tremendous. Without image processing and with the same assumptions as above, 650 man-hours would be needed for a complete visual inspection of a scene of 10 km by 10 km.AcknowledgementThis work has been performed in the scope of the‘airborne minefield detection: pilot project’ led by ITC and co-funded by the European Commission. More information about this project can be found at http://www.itc.nl/ags/projects/demining/.DOVO/SEDEE, the bomb disposal unit of the Belgian army, is in charge of the minefield in Leopoldsburg.Bibliography[1] C. Bruschini and B. Gros, ‘A Survey of Current Sensor Technology Research for the Detection of Landmines’, International Workshop on Sustainable Humanitarian Demining (SusDem'97),29 September-1 October 1997, Zagreb, Croatia [2] J. A. Craib. ‘Mine detection and demining from an operator’s perspective’. Workshop of anti-personnel mine detection and removal, Lausanne,30th June and 1st July 1995[3] Fritzsche, Martin ‘Detection of buried land mines using ground-penetrating radar’. Proc. SPIE Vol. 2496, p. 100-109, Detection Technologies for Mines and Minelike Targets, Abinash C. Dubey; Ivan Cindrich; James M. Ralston; Kelly A. Rigano; Eds, June 1995[4] J.L. van Genderen and B.H.P. Maathuis,‘Airborne Detection of Landmines: a Review of the Techniques and some practical Results’, DGP session, Disarmament and International Security, Regensburg Germany, 26th March 1998.[5] S. Havlík and P. Licko, ‘Humanitarian demining: the challenge for robotic research’, Journal of Humanitarian demining, Issue 2.2, June 1998[6] P.J. Huber, ‘Robust Statistics’, John Wiley & Sons, New York 1981.[7] D. E. Lake, B. Sadler and S. Casey: ‘Detecting regularity in minefields using collinearity and a modified Euclidean algorithm’ Proc. SPIE Vol. 3079, p. 500-507, Detection and Remediation Technologies for Mines and Mine like Targets II, Abinash C. Dubey; Robert L. Barnard; Eds. July 1997[8] McFee, John E.; Ripley, Herb T. ‘Detection of buried land mines using a CASI hyperspectral imager’. roc. SPIE Vol. 3079, p. 738-749, Detection and Remediation Technologies for Mines and Minelike Targets II, Abinash C. Dubey; Robert L. Barnard; Eds. July 1997[9] W. Mees, P. Druyts, D. Borghys, Y. Ouaghli, C.Miravet, J. Santamaria, H. Suess, C. Perneel, M.Acheroy, and J.-L. Valero. ‘Semi-automatic interpretation of airports using multi-sensorinformation ’. IEEE Transactions on Geoscience and Remote Sensing, submitted .[10] K. Scheerer: ‘Airborne multi-sensor system for the autonomous detection of landmines ’. Proc. SPIE Vol. 3079, p. 478-486, Detection and Remediation Technologies for Mines and Mine like Targets II,Abinash C. Dubey; Robert L. Barnard; Eds. July1997Figure 9: mines candidates (crosses) and actual mines (dots) on the visible imageFigure 10: mines candidates (crosses) and actual mines (dots) on the colour infrared image。