Apple harvesting robot picking path planning and(Academy2009)

Apple harvesting robot picking path planning and

simulation

Yuan Yanwei, Zhang Xiaochao, Zhao Huaping

Institute of Mechatronics Technology and Application

Chinese Academy of Agricultural Mechanization Sciences

Beijing, P.R.CHINA

Email: yyw215@https://www.360docs.net/doc/cf5647176.html,

Abstract—For an apple harvestin g robot, the efficien cy of harvestin g main ly depen ds on pickin g path plan n in g. To fi d the best route, this article studied a t colo y algorithm, an d presen ted an improved method in pickin g path optimization. In this athletics methods of picking, with bin ocular cameras, the coordin ates of ripe apples were obtained, the improved ant colony algorithm was used to obtain the optimal method for pickin g. Open Graphics Library was used to simulate the process of pickin g. Test proved that the advanced ant colony algorithm gave better results in fewer simulation loops

Keywords- apple harvesting robot; picking path planning; simulation; virtual reality

I I NTRODUCTION

Multisensor fusion can be used in apple harvesting robots(H all D L et al., 1997), generate the information such as positions of mature apple, the robot body and arm, the environments obtained from multisensors, and so on, to make decisions regarding their control based on the combined information, and avoid obstacles, finding the best route, moving independently, following up the trajectory (Arsenio, A et al., 1998) and so on.

The complex environment and excrescent freedoms of the robot arm (LaValle S M, 2004; Jean F, 2001) make the arm of the harvesting robots hard to control. Thus, the software Multigen Creator has been used to create the model of robot arm. Virtual reality technology has been utilized to simulate the process of picking (Johnson R et al., 2002; Ceres R et al., 1998), which can severe theory for practicality execution, decrease experiment periods and expenditures (Oksanen T, 2004).

II METHODOLOGY

A.The application of ant colony algorithm

Path planning in picking is crucial to the efficiency of apple harvesting robot (Chen P C et al., 1998; Kavraki L E et al., 1994), and the optimization algorithm for picking can be transformed to TSP (K Nagatani, 2002; Laporte G, 1992). An improved ant colony algorithm was studied to solve this problem in this article.

Suppose there are n vertexes (the center of apples) in the undigraph (composed of n apples), M ants in the ant colony, and ?ij(t) is pheromone of edge E(i,j) at t moment, d ij(i,j=1,2,…n) is the distance between the vertexes V i and V j,?ij = 1/d ij is the evaporation rate, then, at t moment, ant k at vertex V i will choose next vertex V j according to formula:

[][]

{}

[][]

[][]

°

°

°

ˉ

°°

°

?

-

≥

<

=

|

?

?

conditions

other

q

q

t

t

q

q

t

V

k

i

k

i

tabu

V

ij

ij

ij

ij

ij

ij

tabu

V

j

,0

,

)(

)(

,

)(

max

arg

β

α

β

α

β

α

η

τ

η

τ

η

τ

(1)

Where proper fraction q is a random digit, while proper fraction q0is a constant, which is generated from simulation of many times, and vertex V j is out of the taboo list of ant k, tabuk composed of apples that ant k passed, and the parameters ? and ? are pheromone and self- elicitation.

After n moments, the ants found a loop containing all the vertexes, and the pheromones of each route should be adjusted according to the following formula:

|

=

Δ

?

+

?

?

=

+M

k

k

ij

ij

ij

t

t

n

t

1

)(

)(

)

1(

)

(τ

ρ

τ

ρ

τ(2)

Where)(t k

ij

τΔis the pheromone deposited at t moment between vertexes V i and V j by ant k,? is pheromone volatilization quotiety, which is a constant between 0 and 1, and:

°ˉ

°

?

-

=

Δ

otherwise

j

i

E

passed

k

ant

if

L

Q

t

k

k

ij

,0

)

,(

,

)(

τ

(3)

Q is a constant, which has little effect on the simulation result if it gives rise to a value that ranges from 100 to 600. L k is the distance that ant k passed throughout all the vertexes.

This calculation was continued until the times NC of loops reached the upper limit times NC max.

B.The improved algorithm

To avoid locally optimal solution, the ant colony algorithm was improved as follows:

°

ˉ

°

?

-

<

Δ

+

?

≥

Δ

+

?

=

+

in

max

)(

)(

)(

)(

)

(

max

min

min

max

m

ij

ij

ij

ij

ij

t

t

t

t

n

t

τ

τ

ρ

τ

τ

ρ

τ

τ

τ

τ

τ

τ

τ

τ

τ

(4)

Where

min

τ and

max

τwere given before the

simulation, and

max

min

τ

τ≤,|

=

Δ

=

Δm

k

k

ij

ij

1

τ

τ.

978-1-4244-4994-1/09/$25.00 ?2009 IEEE

After the completion of each loop, shortest and longest loops were calculated and the data were renewed according to the following formula:

)

()()1()1('

t t t ij ij ij τρτρτΔ?+??=+ (5)Where

°ˉ

°

?-∈∈=Δotherwise L j i E L Q L j i E L Q t l

l s s ij

,0),(,),(,)('

τ (6)

Where

s L is the shortest route and l L is the

longest route of these loops. The improved algorithm can

restrict data in each edge between ?min and ?max, restrict the range of pheromone, and then can decrease the possibility of obtaining a locally optimal solution.

The flow diagram of algorithm is shown in Fig. 1.

Figure 1. Flow diagram of picking path planning

algorithm

C.Virtual Reality Simulations

Two variables were defined to denote the width and length of the area obtained from the DEM file, and a two dimension array was defined to store the elevation data. Initialization must be completed before the program can run. Initially, description table was built for the graph operation, and this was followed to reset pixel format for current picture window, so that it is suitable for display equipment. Simultaneously, pixel format was chosen for equipment description table, and it was made as the current format. Last, in the definition window, draw picture function was used to draw the pictures.

Picking path planning method stated above calculates the optimum route of picking, according to control command, the robot arm executes the picking action through the transform of pan, zoom, and rotation.

In virtual reality system, the robot arm would be controlled by changing six parameters: three position parameters (x ,y , and z ) and three gesture parameters

(pan ,zoom , and rotate ). Suppose the geometrical

transform matrix of three dimension graph is M , and:

?????

o?

????a=4443

42

4134333231242322

21

14131211m m m m m m m m m m m m m m m m M

In the position of transform function, M can be separated into four submatrixes:

??

o

??a=???????

o

???????a?=22211211

4443

42

41

34333231

24232221

14131211M M M M m m m m m m m m m m m m m m m m M #""""###(7)

Where M11 executes zoom, rotation, and shear transform; M12 executes projection transform; M21executes pan transform; and M22 has the function of scale transform for the whole model.

Suppose the coordinates are (x ,y ,z ),Tx is removed along x axis, Ty along y axis, and Tz along z axis, and a new coordinate is obtained (Xm ,Ym,Zm ), then,

T

z y x z

y

x

T

T

m m m T z T y T x T T T z y x Z Y X ???

??

o

?????a+++=??????o????

??a?????o?????a=?

?

??

?o

?????a110100001

00001*11(8)

As to scale transform, a reference point is defined (xf,

yf, zf ); if the model is zoomed (sx, sy, sz ) along x, y , and z axis separately, then, the new coordinate (Xz, Yz, Zz ) can be calculated from the function:

()()()?

?????o????

??a????????o?????a=?

?

???o

?????a1111000000

000

*11f

z f

y f

x z

y x T

T

z z z z s y s x s s s s z y x Z Y X (9)

The arithmetic stated above was used to execute the transform of robot and scene model. Through the matrix transformation, the robot arm was moved from one apple to another according to the route that was planned.

OpenGL provided responding transform function. During the virtual reality simulation, a timer is necessary. For each step, the timer will generate a message, and the position and gesture of robot components and scene model were calculated, thus the steering of the system was realized in virtual reality circumstances.

Fig. 2 shows the image of simulated robot arm that moved from (a) to (b), (c), and (d) in virtual reality platform.

Figure 2. simulated robot arm in virtual reality

platform

III R ESULTS AND DISCUSSION

Images of projected 2D coordinates of 130 and 150 ripe apples were figured out and plotted in Fig. 3 (a) and Fig. 3 (b).

Figure 3. (a) Projected coordinates of 130 apples

Figure 3. (b) Projected coordinates of 150 apples

By giving different values to the parameters, and executing the calculation with standard and advanced ant colony algorithm, the results were obtained as shown in table 1.

Table 1ˊ Parameters of each exercise for 130 apples path planning

Serial number

Periodicities of find optimum results Shortest distance Standard algorithm Advanced algorithm Standard algorithm Advanced algorithm

1 426 635 38487.6 38329.7

2 262 318 37403.8 36542.9

3 137 222 11808.6 10903.3

4 13

5 218 11823.7 11068.4 5 120 195 7156.4 6959.3

6 429 128 7020.

7 6721.2 7 69 45 6706.0 6643.5

8 5 5 7134.

9 6851.4 9 NULL NULL

The number of ants is the key to the algorithm; when the number is big, the ability of global search is strong,

but the calculation speed is lowered. Through many simulation experiments, we know that the number of ant is 3/4 of apple is best.

Ant K will decide where to move according to pheromone level τ, and the heuristics reflect the importance of the information that accumulated when ants move. The parameter ? has dualism effect on the algorithm. When ? is too big, the astringency is low, and if ? is too small, the possibility of local optimization will increase. According to many experiments, it has been found that better results can be obtained if the parameters are set as follows:

?=1,?=5;10,01.0max min ==ττ;[]8.0,5.0∈ρ.When standard ant colony algorithm was used in 130 apples image, best value of 6706.0 millimeters was obtained after 69 calculation periodicities, the optimization routes are shown in Fig. 4 (a). When advanced ant colony algorithm was used, best value of 6643.5 millimeters was obtained after 45 calculation periodicities, and the optimization routes are shown in

Fig. 4 (b).

Figure 4. (a) Optimal path of 130 apples obtained by

standard ant colony algorithm

Figure 4. (b) Optimal path of 130 apples obtained by

advanced ant colony algorithm

Based on the above data, it can be concluded that advanced ant colony algorithm gives better results in fewer simulation loops.

CONCLUSIONS

Through the study of arithmetic, programming, and experimentation, the following conclusions are derived: 1) Apple harvesting robots are different from industrial robots in path planning; path information is collected before harvesting, and according to this path, the robots navigate;

2) Advanced ant colony algorithm has been proposed to plan the picking path, and it has generated optimal solutions;

3) The virtual reality platform has driven the model and the virtual scene, and has fulfilled the simulation of picking steering system.

ACKNOWLEDGMENTS

The authors thank the members of CAAMS for their help. The projects were supported by Chinese National High-tech R&D Programs (No. 2006AA10A305, and No. 2006AA10Z254) and Chinese Mega-projects of Science Research for the 10th Five-Year Plan (No. 2006BAD11A01).

REFERENCES

[1]Arsenio, A. and Ribeiro, M. I.: Active range sensing for mobile

robot localization, Proc. - IEEE/RSJ Int. Conf. Intell. Robots.

Syst. 2. 1998: 1066-1071.

[2]Burdea G, Coiffed P. Virtual Reality Technology [M]. New

York: John Wiley and Sons, Inc., 1994.

[3]Ceres R, Pons F L, Jimenez A R, et al. Design and

implementation of an aided fruit-harvesting Robot [J]. Industrial

Robot, 1998, 25(5):337-346.

[4]Chen P C, H ang Y K. SANDROS: a dynamic graph search

algorithm for motion planning [J]. IEEE Transactions on

Robotics and Automation. 1998, 14(3): 390-403.

[5]Hall D L, Linas J. An introduction to multi-sensor data fusion [J].

Proceeding of the IEEE?1997?85?1??6 - 23.

[6]Jean F. Complexity of Nonholonomic Motion Planning [J].

International Journal of Control, 2001, 74(8).

[7]Johnson R., Z.Qu, S.Jayaram, et cl. Autonomous Spacecraft

Vehicle H ealth Monitoring and Control System Based on

Real-Time Model-Based Simulation. Submitted for publication,

2002.

[8]Kavraki L E, Latombe J C. Randomized preprocessing of

configurations space for fast path planning [C]. Proceedings

IEEE International Conference on Robotics and Automation,

San Diego, CA, USA, 1994: 2138-2145.

[9]K Nagatani, I H irayama et al. Motion Planning for Mobile

Manipulator with Keeping Manipulability [C]. In: Proceeding of

the 2002 IEEE/RSJ International Conference on Intelligent

Robots and Systems. 2002: 1663-1668.

[10]Laporte G. The TSP: an Overview of Exact and Approximate

Algorithm [J]. European Journal of Operation Research. 1992,

59 (1): 231-247.

[11]LaValle S M. Planning Algorithms [M]. University of Illinois,

2004.

[12]Lu H eng, Jiang Nan. Three Dimensional Showing of Terrain

Based on the OpenGL and Relief MAP [J]. Computer

Engineering(in Chinese), 2004(4): 174-177.

[13]Oksanen, T., Visala, A. Optimal control of tractor-trailer system

in headlands. ASAE International Conference on Automation

Technology for Off-road Equipment, Kyoto. Japan. 2004:

255-263.

[14]Reinelt G. The Traveling Sales Man: Computational Solutions

for TSP Applications [M]. Lecture Notes in Computer Science,

Berlin: Springer Verlag(in Chinese), 1994: 840. [15]Sun Min, Xue Yong, Ma Ai-nai. 3D Visualization of Large DEM

Set Based on Grid Division [J]. Journal of Computer-Aided Design & Computer Graphics (in Chinese), 2002, 14(6): 566 - 570.

[16]Wright R S, Sweet M J. OpenGL super bible [M]. Xiaoxing

Studio Transl. Beijing: People's Post and Telecommunications Publishing House, 2001: 264-270.