Adapting Underwater Physical Link Parameters Using Data Driven Algorithms D. Melani Jayasuriya BSc Eng. (Hons), University of Moratuwa A THESIS SUBMITTED FOR THE DEGREE OF MASTER OF ENGINEERING DEPARTMENT OF ELECTRICAL AND COMPUTER ENGINEERING NATIONAL UNIVERSITY OF SINGAPORE 2010 Acknowledgements I would like to thank Assistant Professor Mandar Chitre, for his supervision and support during this research. I am grateful to him for having been constantly supportive and encouraging, and for his helpful suggestions and critical comments. It was tough at times to meet up with your expectations, but I should say it truly helped. First of all I should thank Professor Rohan Abeyratne, Director of SingaporeMIT Alliance for Research and Technology (SMART) and Dr. David Burke, then Program Manager of center for environmental sensing and modeling (censam) under SMART for introducing me to my supervisor. I am truly grateful for their help at the time. I had the pleasure of working with people in the Acoustic Research Laboratory (ARL), Tropical Marine Science Institute (TMSI) who always treated me as part of ARL family and always greeted with a warm smile. I should especially mention Satish Shankar for sharing his knowledge on the problem. I would like to thank Dr. Paul Seekings and Kian Peen of Marine Mammal Research Laboratory and Associate Professor Lonce Wyse of Interactive and Digital Media Institute for their support and encouragement. I would also like to thank the wonderful people at Republic Polytechnic, for their understanding and support during this period. I was able to attend all my evening lectures for two semesters, two days a week because of their kind support. Not many employers would have have done that, and I am deeply thankful to all i ii of them. I am truly grateful for the support and understanding of Dr. Wong Hau Shian for her continuous encouragement that has helped me in completing the research as well as the seminar requirements. Dr. Zhou Kainan deserves special thanks for the many useful discussions we had and for her valuable advice and help throughout.I am grateful to have colleagues like Mr.Tan Kok Cheng and Ms.Yap Woan Leng who helped me manage my work, specially during the stressful period of thesis submission. To my friends, especially Nihari, Erandi, Ganthi and Natasha, thank you for your encouragement and the concern you have shown throughout this period. Finally, I am truly grateful to my family for their encouragement, motivation and love. My dear mom and dad, for the endless times you stood by me, believed in me and guided me to success. Your words of wisdom, continuous encouragement and heartiest blessings have always my biggest strength. My dear little brother, for the thousands of times you did the dishes for me, and for all the times you were there for me in good times and bad. My dear husband, who deserves credit on top of all, for spending many hours day through night helping to simulate and analyze the results as well as proof-reading and formatting the thesis. I am sure you now know more about this topic now than you ever wanted to. Your endless moral support and words of wisdom did not ever let me give up no matter how much I wanted to. I am truly blessed to have you all as my family. Your love and support have brought me this far and I will never let you down. Contents Acknowledgements i Summary v List of Figures vii List of Tables ix Abbreviations x List of Notations Introduction 1.1 Background and Motivation . 1.2 Problem Statement . . . . . . 1.3 Choice of Modem Parameters 1.4 Explore or Exploit? . . . . . . 1.5 Thesis Contributions . . . . . 1.6 Thesis Organization . . . . . . xi . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 Background and Related Work 2.1 Multi Armed Bandit Problem . . . . . . . . . . . . . . . . . . . . . 2.2 Application of Multi Armed Bandit to the Problem . . . . . . . . . 2.3 Gittins Index Strategy . . . . . . . . . . . . . . . . . . . . . . . . . 12 Problem Formulation 15 3.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 3.2 Channel Representation . . . . . . . . . . . . . . . . . . . . . . . . 20 Preliminary Solution Strategies 4.1 Basic Strategies . . . . . . . . . . . 4.1.1 Brute Force Strategy . . . . 4.1.2 First Hit Strategy . . . . . . 4.1.3 Random Selection Strategy 4.1.4 -Greedy Strategy . . . . . . iii . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 23 24 28 31 33 Contents 4.2 Enhanced Strategies . . . . . . . . . . 4.2.1 Enhanced Brute Force Strategy 4.2.2 Enhanced First Hit Strategy . . 4.2.3 Enhanced -greedy Strategy . . iv . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ranked Exploration Strategy 5.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Weights Matrix . . . . . . . . . . . . . . . . . . . . . . . 5.3 The Algorithm . . . . . . . . . . . . . . . . . . . . . . . 5.4 Likelihood Assignment . . . . . . . . . . . . . . . . . . . 5.4.1 Proportional Likelihood Assignment . . . . . . . . 5.4.2 Rank-based Likelihood Assignment . . . . . . . . 5.5 Adaptive Temperature Profile of Probability Distribution 5.5.1 Simulated Annealing . . . . . . . . . . . . . . . . 5.6 Selection using ‘Roulette Wheel Selection’ . . . . . . . . Results and Discussion 6.1 Simulation formulation . . . . . . . . . 6.2 Brute Force Strategy vs. Enhancement 6.3 First Hit Strategy vs. Enhancement . . 6.4 Random Selection Strategy . . . . . . . 6.5 -Greedy Strategy . . . . . . . . . . . . 6.6 Enhanced -Greedy Strategy . . . . . . 6.7 Ranked Exploration Strategy . . . . . 6.8 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 38 39 41 . . . . . . . . . 43 43 45 45 48 48 49 50 51 54 . . . . . . . . 56 56 58 60 62 63 65 67 69 Conclusion 75 7.1 Summary of Results . . . . . . . . . . . . . . . . . . . . . . . . . . 75 7.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 A MATLAB codes for generating data sets 78 A.1 Generation of data sets . . . . . . . . . . . . . . . . . . . . . . . . . 78 A.2 Sorting the data sets . . . . . . . . . . . . . . . . . . . . . . . . . . 80 References 83 Summary This thesis addresses the question of transferring a file in minimum possible time using an underwater acoustic link that can be tuned by changing physical link parameters. Assuming we have no prior knowledge about the average data rate resulting from any of the parameter choices, we have to decide between exploring for new parameter values versus exploiting the best from the known parameter values. Hence our objective is to devise a strategy to balance this exploration and exploitation in order to transfer a file in minimum time. In the process of finding an optimal solution, several data driven algorithms such as Brute Force, First Hit, Random Selection and -Greedy were studied primarily and the effect of performance for varying search space, burst size and file size for each algorithm were investigated. When they did not produce promising results, we moved on to exploring our own strategies and enhancing the available strategies with the contextual information available. Enhanced -Greedy is one such example. Learning from widely accepted theories of optimization, such as Simulated Annealing and Rank-based assignments, the proposed Ranked Exploration Strategy was formulated. It does not have a fixed probability to explore, but rather it has a distribution from which it decides whether to explore or to exploit. And this distribution is not fixed either. The more confident we become about the observations made, the more biased the distribution becomes towards exploitation. This was also analyzed on its performance with respect to the various parameters. Summary vi Simulations were performed on channel data matrices which effectively model the underwater acoustic environment. Simulation results showed that the Ranked Exploration performed well while providing a computationally efficient solution. List of Figures 3.1 PER vs. BER for lj = 8000 bits and h = 2000 bits . . . . . . . . . 21 3.2 Channel Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 4.1 Throughput of Brute Force Strategy - Analytical Results vs. Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 4.2 Throughput of Random Selection Strategy - Analytical Results vs. Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 5.1 Flowchart of Ranked Exploration Algorithm . . . . . . . . . . . . . 46 5.2 Sample Proportional Likelihood Assignment . . . . . . . . . . . . . 49 5.3 Principle of Simulated Annealing . . . . . . . . . . . . . . . . . . . 51 5.4 Temperature profiles for different values of λ . . . . . . . . . . . . . 53 5.5 Comparison of throughput with search spaces with and without temperature profiling for Ranked Exploration Strategy . . . . . . . 54 5.6 An instance of Roulette Wheel Selection for data in table 5.2 . . . . 55 6.1 Variation of throughput with search space for Brute Force Strategy 6.2 Comparison of throughput with search space for Brute Force and Enhanced Brute Force . . . . . . . . . . . . . . . . . . . . . . . . . 59 6.3 Variation of throughput with search space for First Hit Strategy . . 60 6.4 Comparison of throughput with search space for First Hit and Enhanced First Hit . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 6.5 Variation of throughput with search space for Random Selection Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 vii 59 List of Figures viii 6.6 Variation of throughput with search space for different for Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 6.7 Variation of throughput with file sizes for different for -Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 6.8 Variation of throughput with burst sizes for different for -Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 6.9 Variation of throughput with search space for different for Enhanced -Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . 66 6.10 Variation of throughput with file sizes for different for Enhanced -Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 6.11 Variation of throughput with burst sizes for different for Enhanced -Greedy Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 6.12 Variation of throughput with search space for Ranked Exploration Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.13 Variation of throughput with file sizes for Ranked Exploration Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 6.14 Variation of throughput with burst sizes for Ranked Exploration Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 6.15 Comparative throughput for various strategies . . . . . . . . . . . . 70 6.16 Comparative throughput for various algorithms with varying search spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 6.17 Comparative throughput for various algorithms with varying file sizes 72 6.18 Comparative throughput for various algorithms with varying burst sizes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 List of Tables 1.1 Average data rate for various link and coding schemes. . . . . . . . 5.1 A sample set of Selection Probabilities for Proportional Likelihood Assignment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 5.2 Selection Probabilities (according to a simple temperature profile) for a sample set of likelihoods and ranks . . . . . . . . . . . . . . . 55 ix Chapter 6. Results and Discussion 70 is set at 200 bits per second, which is around 70% of the mean of average data rate for the given data sets. 1200 1000 Throughput (bps) 800 600 400 200 −200 eBF eFH RS Strategies eGR RE Figure 6.15: Comparative throughput for enhanced Brute Force (eBF), enhanced First Hit (eFH), Random Selection (RS), -Greedy (eGR) and Ranked Exploration Strategy (RES) algorithms for a 5Mb file transfer with m = 25 and n = 33. Figure 6.15 shows the mean throughput with 95% confidence intervals for a typical simulation instance with F = 5Mb over a 25×33 channel. Figure 6.15 shows that Ranked Exploration Strategy has the highest mean throughput, followed by enhanced -Greedy, enhanced First Hit, Random Selection and enhanced Brute Force. Figure 6.16 shows the variation of throughput with search space for a 5Mb file. The throughput decreases as the search space increases since the ‘good’ link Chapter 6. Results and Discussion 71 Throughput (bps) 1500 eBF eFH RS eGR RE 1000 500 25x33 50x33 100x33 Search Space 150x33 200x33 Figure 6.16: Comparative throughput for Ranked Exploration Strategy (RES), Enhanced Brute Force (eBF), Random Selection (RS), Enhanced First Hit (eFH) and Enhanced -Greedy (eGR) algorithms with search space mn for 5Mb file and burst size of 100. schemes become increasingly harder to locate. Ranked Exploration Strategy beats all the rest of the strategies. Figure 6.17 shows the variation of throughput with file size for a 25 × 33 channel. It is evident that for relatively small file sizes, the throughput curves are close to each other, confirming the intuition that if a file is very small, random selection is as good a strategy as any other. In fact, for very small file sizes, it is indeed inefficient to explore and therefore the Ranked Exploration strategy performs as poor as the Random Selection or the First Hit algorithms. For Enhanced Brute Force strategy, transmissions of file sizes below 5Mb exhausts during exploration phase. Therefore these transmissions not benefit from the exploitation of best data rates, thus displaying results which heavily rely on the channel distribution. As a result, true benefits of enhanced Brute Force can be analyzed only above file sizes of 5Mb for a burst size of 100 for the given channel distributions. The key Chapter 6. Results and Discussion 72 1400 1200 Throughput (bps) 1000 eBF eFH RS eGR RE 800 600 400 200 10 10 10 10 File Size (bits) 10 10 Figure 6.17: Comparative throughput for Ranked Exploration Strategy (RES), Enhanced Brute Force (eBF), Random Selection (RS), Enhanced First Hit (eFH) and Enhanced -Greedy (eGR) algorithms with file size for search space 25 × 33 and burst size of 100. point to note from Figure 6.17 is that as the file size increases, Ranked Exploration shows considerable performance gains over the rest. Figure 6.18 shows the variation of throughput with burst size for a 5Mb file over a 25× 33 channel. In general, the curves follow a bell shape with a maxima. This follows from the fact that transmitting very small burst sizes is not efficient due to the protocol overhead. Transmitting very large bursts is not efficient either since a lot of time may be lost if a large data burst fails to get successfully received. The figure shows that enhanced Brute Force deviates from the rest in its behaviour against varying burst sizes. This is due to the fact that smaller burst Chapter 6. Results and Discussion 73 1500 eBF eFH PR eGR BI RE Throughput (bps) 1000 500 10 10 10 Burst Size Figure 6.18: Comparative throughput for Ranked Exploration Strategy (RES), Enhanced Brute Force (eBF), Random Selection (RS), Enhanced First Hit (eFH) and Enhanced -Greedy (eGR) algorithms with burst size for 5Mb file and search space 25 × 33 sizes allow more room for exploitation of the best data rate. Towards the higher burst sizes, the file size 5Mb exhausts during exploration thereby ending up in throughput results which heavily rely on the distribution of the channel model. Since only a portion of the channel model is explored by the time the file exhausts, the throughput heavily relies on the PERs of those attempted schemes. Most of the data sets in our simulation setup not seem to have the worst PERs among the first few schemes, therefore resulting in an increased throughput as shown in figure 6.18. Comparative results among different strategies show that Ranked Exploration Strategy, the method proposed in this thesis, is able to provide a solution which Chapter 6. Results and Discussion 74 exceeds the performance of other strategies. The processing time of Ranked Exploration displays a mean of 0.02 seconds with a standard deviation of 0.01 seconds to transmit a 5Mb file with search space 200 × 33 and burst size of 100 when simulations were run on MATLAB version running on an Intel Core i7 620M processor. This could be further improved on an actual implementation, thus making itself a good candidate for real time, time critical applications. Chapter Conclusion This thesis has presented the systematic development of data driven algorithms in order to tune the underwater physical link parameters. A summary of the findings is given in the first part of this chapter. A discussion of future directions that warrant more study concludes the chapter and the thesis. 7.1 Summary of Results Our initial problem was to transfer a finite sized file in minimum possible time using an underwater acoustic link. Assuming we have no prior knowledge about the average data rate resulting from any of the parameter choices, we were to decide between exploring for new parameter values versus exploiting the best from the available parameter values. Hence our objective was to devise a strategy to balance this exploration and exploitation in order to transfer a file in minimum time. 75 Chapter 7. Conclusion 76 In the process of finding an optimal solution, several existing basic strategies were studied. When they did not produce promising results, we moved on to coming up with our own strategies and enhancing the available strategies with the contextual information available. Enhanced -Greedy was one such example. Finally learning from widely accepted theories of optimization, such as Simulated Annealing and Rank-based assignments, we managed to come up with a totally new strategy which we called as Ranked Exploration Strategy. As the name implies, it didn’t have a fixed probability to explore, but it rather had a temperature pro- file from which it decided whether to explore or to exploit. And this temperature profile was not fixed either. The more confident we become about the observations made, the temperature profile becomes more biased towards exploitation, unlike initially when it favors exploration more. Twenty channel matrices were generated for dimensions 25 × 33, 50 × 33, 100×33, 150×33 and 200×33 (see Appendix A) in order to model the underwater acoustic channel. Simulations were carried out on these channel matrices with 100 independent trials on each, for varying parameters such as search space, file size and burst size. The simulations were performed on all the strategies we have discussed in this thesis, and the results were plotted as shown in Chapter 6. Observations from the generated results show that Ranked Exploration Strategy, with its improved performance, performs best in terms of throughput compared to other strategies. Chapter 7. Conclusion 7.2 77 Future Work We have illustrated various possible data driven algorithms to tune physical layer parameters. All these algorithms presented in this thesis have made use of basic channel statistics in the form of bit error rate in packet transmissions. However, it may be possible to get better performance by exploiting other higher order channel statistics or models. This could be explored further in a future project. Further studies can be carried out to examine what combinations of channel statistics will produce best results. The work presented in this thesis relies on the strong assumption that the channel will remain static throughout the course of the file transfer. However in a realistic setting, the channel might tend to vary over the long transfer times, especially for larger file sizes. We have not studied the performance of the strategies under such circumstances, as it is not within the scope of this thesis. It would be an interesting area to explore in future. Further studies could also look into ways of enhancing the Ranked Exploration strategy in order to get better performance in future. With these results, the next step will be to perform actual experiments in underwater in order to investigate the performance even further. The algorithms discussed in this thesis are independent of the physical layer implementation, and can easily be implemented as link tuners for any modem with tunable parameters. Appendix A MATLAB codes for generating data sets A.1 Generation of data sets %Random c h a n n e l g e n e r a t o r %M o d i f i c a t i o n : Keep a c o n s t a n t number o f good schemes , 15 i n t h i s c a s e f u n c t i o n [ genData genTruePER ] = g e n _ r a n d o m _ c h a n n e l ( ) cRows = 25; %C o r r e s p o n d s t o l i n k schemes cColumns = 3 ; %C o r r e s p o n d s t o c o d e s / p a c k e t s i z e s maxTrials = ; %T o t a l number o f c h a n n e l s vGoodSchemes =15; DRlow =2000; DRhi =3000; %g e n e r a t i n g c h a n n e l f o r trials = : maxTrials f o r row = : cRows xPoint = randint ( , , [ , cColumns ] ) ; i f xPoint1 f o r column = : xPoint − genTruePER ( row , column , trials ) = ; end genTruePER ( row , xPoint , trials ) = rand ; f o r column = xPoint +1: cColumns genTruePER ( row , column , trials ) = ; end end i f xPoint == f o r column = : cColumns genTruePER ( row , column , trials ) = ; 78 Appendix A. MATLAB codes for generating data sets 79 end genTruePER ( row , xPoint , trials ) = rand ; else i f xPoint == cColumns f o r column = : cColumns −1 genTruePER ( row , column , trials ) = ; end genTruePER ( row , xPoint , trials ) = rand ; end end end end f o r trials = : maxTrials f o r time = : cRows genData . linkRate ( trials , time ) = randint ( , , [ DRlow DRhi ] ) ; %For t h e ← s a k e o f m a i n t a i n i n g data . time a s a row v e c t o r . end end %Adding i n t h e good s t u f f f o r trials =1: maxTrials f o r i =1: f l o o r ( normrnd ( vGoodSchemes , ) ) b e s t schemes goodRow=randint ( , , [ , cRows ] ) ; xPoint=f l o o r ( normrnd ( , ) ) ; i f xPoint cRows xPoint=cRows ; end end i f xPoint1 f o r column = : xPoint − genTruePER ( goodRow , column , trials ) = ; end genTruePER ( goodRow , xPoint , trials ) = rand ; f o r column = xPoint +1: cColumns genTruePER ( goodRow , column , trials ) = ; end end i f xPoint == f o r column = : cColumns genTruePER ( goodRow , column , trials ) = ; end genTruePER ( goodRow , xPoint , trials ) = rand ; else i f xPoint == cColumns f o r column = : cColumns −1 genTruePER ( goodRow , column , trials ) = ; end genTruePER ( goodRow , xPoint , trials ) = rand ; %These a r e t h e ← Appendix A. MATLAB codes for generating data sets 80 end end end end end A.2 Sorting the data sets %G e n e r a t e s code r a t e , p a c k e t s i z e and data r a t e and s o r t s them i n %d e c r e a s i n g o r d e r o f data r a t e . I n p u t i s t h e t o t a l number o f p a c k e t s i z e s %t o be g e n e r a t e d . %Note : e n s u r e t h e s i z e o f t h e o v e r h e a d i s same a s t h o s e used i n %s i m u l a t i o n s , e l s e t h e s o r t i n g w i l l be i n c o r r e c t f u n c t i o n [ sort edCodeRa te s o r t e d P a c k e t S i z e s ortedDa taRate ] = genS ortedStu ff ( ← packetSizeOptions ) overhead = 0 ; linkDataRate = e3 ; maxCodedPacketSize = 4000; %I f coded p a c k e t s i z e e x c e e d s t h i s l i m i t s , t h a t ← p a c k e t s i z e / code o p t i o n i s e l i m i n a t e d minPacketSize = ; maxPacketSize = 0 ; i f n a r g i n==0 packetSize = [ , 0 , 0 , 0 , 0 , 0 , 0 ] ; %d e f a u l t p a c k e t s i z e ← options else p a c k e t S i z e I n t e r v a l = ( maxPacketSize − minPacketSize ) / p a c k e t S i z e O p t i o n s ; f o r i = : packetSizeOptions packetSize ( i ) = minPacketSize + ( i −1) ✯ p a c k e t S i z e I n t e r v a l ; %← generate packet s i z e s end end codeRate = [ / , / , / , / , / ] ; %s t r o n g e r c o d e s i n t h e end %L e t s f i x Eb/N0 = ber = [ , 1e −3 , 1e −1 , . , ] ; %r e s u l t a n t BER a f t e r c o d i n g ber = ber . ✯ 0 ; f o r codeRateIndex = : numel ( codeRate ) f o r p ac ke tS i ze In d ex = : numel ( packetSize ) per ( codeRateIndex , p ac ke tS i ze In de x ) = − ( − ber ( codeRateIndex ) ) ˆ← packetSize ( p ac ke tS i ze In de x ) ; end end Appendix A. MATLAB codes for generating data sets 81 f o r codeRateIndex = : numel ( codeRate ) f o r p ac ke tS i ze In d ex = : numel ( packetSize ) dataRate . dataRate ( codeRateIndex , p ac ke tS i ze In d ex ) = linkDataRate ✯ ← codeRate ( codeRateIndex ) ✯ . ( packetSize ( p ac ke tS i ze In d ex ) / ( packetSize ( p ac ke tS i ze In de x )+overhead ) ) ← ; dataRate . codeRate ( codeRateIndex , p ac ke tS i ze In d ex ) = codeRate ( ← codeRateIndex ) ; dataRate . packetSize ( codeRateIndex , p ac ke t Si ze In d ex ) = packetSize ( ← p ac ke tS i ze In de x ) ; end end i = 1; f o r codeRateIndex = : numel ( codeRate ) f o r p ac ke tS i ze In d ex = : numel ( packetSize ) %p e r L i n e a r ( i ) = p e r ( codeRateIndex , p a c k e t S i z e I n d e x ) ; data RateLine ar ( i ) = dataRate . dataRate ( codeRateIndex , p ac ke tS i ze In de x ) ; code RateLine ar ( i ) = dataRate . codeRate ( codeRateIndex , p ac ke tS i ze In de x ) ; p a c k e t S i z e L i ne a r ( i ) = dataRate . packetSize ( codeRateIndex , p ac ke tS i ze In de x← ); i = i +1; end end %S o r t i n g t h e FEC/ p a c k e t s i z e c o m b i n a t i o n s i n d e c r e a s i n g o r d e r o f data r a t e [ sortedVector sortedIndex ] = s o r t ( dataRateLinear , ✬ descend ✬ ) ; f o r i = : numel ( data RateLine ar ) f o r j = : numel ( dataR ateLine ar ) i f sortedVector ( i )==data RateLine ar ( j ) sort edCodeRa te ( i ) = code RateLine ar ( j ) ; s o r t e d P a c k e t S iz e ( i ) = p a c k e t S i z e L i n e ar ( j ) ; sort edDataRa te ( i ) = data RateLine ar ( j ) ; end end end %e l i m i n a t i n g e n t r i e s which r e s u l t i n p a c k e t s b l o a t e d beyond l i m i t s f o r i = : numel ( sort edCodeRa te ) i f sort edCodeRa te ( i ) ✯ s o r t e d P a c k e t Si z e ( i ) > m a x C o d e d P a c k e t S i z e sort edCodeRa te ( i ) = ; s o r t e d P a c k e t Si z e ( i ) = ; end end j =1; f o r i = : numel ( sort edCodeRa te ) i f sort edCodeRa te ( i ) ˜=0 T so rt ed C od eR a te ( j ) = sort edCodeRa te ( i ) ; T s o r t e d P a c k e t S i z e ( j ) = s o r t e d P a c k e t S i ze ( i ) ; T so rt ed D at aR a te ( j ) = sort edDataRa te ( i ) ; j = j +1; end end sort edCodeRa te = T so rt ed C od eR at e ; Appendix A. MATLAB codes for generating data sets s o r t e d P a c k e t S i ze = T s o r t e d P a c k e t S i z e ; sort edDataRa te = T so rt ed D at aR at e ; end 82 References [1] P. Ozog, M. Leeser, and M. Stojanovic, “Adapting the usrp as an underwater acoustic modem,” in Proceedings of Thirteenth Annual Workshop on HighPerformance Embedded Computing (HPEC2009), September 2009. [2] E. M. Sozer, M. Stojanovic, and J. G. Proakis, “Underwater acoustic networks,” IEEE Journal of Oceanic Engineering, vol. 25, no. 1, pp. 72–83, January 2000. [3] D. Kilfoyle and A. Baggeroer, “The state of the art in underwater acoustic telemetry,” IEEE Journal of Oceanic Engineering, vol. 25, no. 1, pp. 4–27, Jan. 2000. [4] A. S. Rao and R. Kumar, “Implementation of adaptive filter algorithm for underwater acoustic system,” International Journal of Recent Trends in Engineering, vol. 2, no. 2, November 2009. [5] R. Jurdak, P. Baldi, and C. V. Lopes, “Software-driven sensor networks for short-range shallow water applications,” Ad Hoc Netw., vol. 7, no. 5, pp. 837–848, 2009. [6] J. Preisig, “Acoustic propagation considerations for underwater acoustic communications network development,” in Proceedings of WUWNet06, California, September 2006. [7] B. Borowski and D. Duchamp, “Short paper: The softwater modem a software modem for underwater acoustic communication,” in Proceedings of WUWNet09, California, November 2009. [8] M. Chitre, S. H. Ong, and J. Potter, “Performance of coded ofdm in very shallow water channels and snapping shrimp noise,” in Proceedings of IEEE/MTS OCEANS05, Washington DC, September 2005. 83 Reference 84 [9] S. Shankar, M. Chitre, and M. Jayasuriya, “Data driven algorithms to tune physical layer parameters of an underwater communication link,” in Proceedings of OCEANS’10, Syndey, Austrailia, May 2010. [10] Z. Wei-Qing, W. Chang-Hong, Z. M. Pan Feng, W. Rui, Z. Xiang-Jun, and D. Yong-Mei, “Underwater acoustic communication system of auv,” in Proceedings of IEEE/MTS OCEANS98, France, 1998. [11] S. Guo and Z. Zhao, “Design of a qpsk-cdma acoustic communication system for multiple underwater vehicles,” in Proceedings of ICMA 2009, China, 2009. [12] H. Robbins, “Some aspects of the sequential design of experiments,” Bulletin of the American Mathematical Society, vol. 55, pp. 527–535, 1952. [13] A. Mahajan and D. Teneketzis, “Multi-armed bandit problems.” [14] A. O. Hero, D. A. Castan, D. Cochran, and K. Kastella, Foundations and Applications of Sensor Management, 1st ed. Springer Publishing Company, Incorporated, 2007. [15] R. Bellman, Dynamic Programming. Dover Publications, 2003. [16] W. B. Powell, Approximate Dynamic Programming: Solving the Curses of Dimensionality (Wiley Series in Probability and Statistics). Wiley-Interscience, 2007. [17] J. C. Gittins, Multi-armed Bandit Allocation Indices. Mathematical Institute, University of Oxford: John Wiley and Sons, 1988. [18] J. Ni no-Mora, “Computing an index policy for multiarmed bandits with deadlines,” in ValueTools ’08: Proceedings of the 3rd International Conference on Performance Evaluation Methodologies and Tools. ICST, Brussels, Belgium, Belgium: ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), 2008, pp. 1–10. [19] P. Whittle, “Restless bandits: Activity allocation in a changing world,” Journal of Applied Probability, vol. 25, pp. 287–298, 1988. [20] K. D. Glazebrook, D. Ruiz-Hernandez, and C. Kirkbride, “Some indexable families of restless bandit problems,” Advances in Applied Probability, vol. 38, no. 3, pp. 643–672, 2006. Reference 85 [21] J. Ni no-Mora, “A marginal productivity index policy for the finite-horizon multiarmed bandit problem,” in Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 2005, December 2005. [22] M. A. Chitre and M. Motani, “On the use of rate-less codes in underwater acoustic file transfers,” in Proceedings of Oceans 2007 Europe International Conference, Aberdeen, Scotland, vol. 1-3, June 2007, pp. 457–462. [23] C. Watkins, “Learning from delayed rewards,” Ph.D. dissertation, University of Cambridge,England, 1989. [24] J. Vermorel and M. Mohri, “Multi-armed bandit algorithms and empirical evaluation,” in In European Conference on Machine Learning. Springer, 2005, pp. 437–448. [25] S. Kirkpatrick, C. D. G. Jr., and M. P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, pp. 671–680, 1983. [26] R. E. Burkard and F. Rendl, “A thermodynamically motivated simulation procedure for combinatorial optimization problems,” European Journal of Operations Research, vol. 17, pp. 169–174, 1984. [27] D. S. Johnson, C. R. Aragon, L. A. McGeoch, and C. Schevon, “Optimization by simulated annealing: an experimental evaluation. part i, graph partitioning,” Operational Research, vol. 37, no. 6, pp. 865–892, 1989. [28] E. Bonomi and J. Lutton, “The N-city travelling salesman problem: statistical mechanics and metropolis algorithm,” SIAM Review, vol. 26, no. 4, pp. 177– 193, 1984. [29] E. Bonomi and J.-L. Lutton, “The asymptotic behaviour of quadratic sum assignment problems: A statistical mechanics approach,” European Journal of Operational Research, vol. 26, no. 2, pp. 295–300, August 1986. [...]... knowledge, and remaining file size 1.5 Thesis Contributions We present a data driven approach for tuning the physical layer parameters of a communication link to optimize data rates, assuming the channel remains static over the course of a file transfer Our approach does not need any knowledge of the physics of the channel Several data driven algorithms such as Brute Force, First Hit, Random Selection and -Greedy... fine tune their parameters during operation The objective for tuning the parameters could be to optimize data rates, protect against errors, minimize power, and so on If the physics of the channel is completely known, it is possible to determine the values of these parameters for a given objective However in practice, it is quite difficult to know the state of the channel completely The parameters usually... options as link schemes and coding schemes With link schemes, we refer to all tunable parameters of a communication system excluding FEC coding and packet size And the coding schemes are FEC coding and packet size combinations At any given time, the user is to choose a link- coding scheme combination to transfer the file Hence, our problem can be considered as a multi-armed bandit problem where the link- coding... minimized 3.1 Definitions 1 Choice of link schemes, FEC codes, and payload sizes: Let the size of the file to be transferred be F We define the n-tuple consisting of all tunable parameters of a communication system excluding FEC coding and packet size as a link scheme Let the communication modem have m link schemes available, each of them resulting in an uncoded link rate of di Chapter 3 Problem Formulation... Introduction 4 Table 1.1: Average data rate for various link and coding schemes Modulation 2-DPSK 2-DPSK 2-DPSK 4-DPSK 4-DPSK 4-DPSK 8-DPSK 8-DPSK 8-DPSK 8-DPSK Code Rate 1/2 1/3 1/4 1/3 1/4 1/6 1/4 1/8 1/12 1/16 BER Avg Data Rate (bps) 1.2 × 10−2 0 −4 3.2 × 10 137 < 10−4 449 −2 1.6 × 10 0 1.5 × 10−3 0.012 < 10−4 399 −2 7.7 × 10 0 2.9 × 10−2 0 4.3 × 10−4 10 −4 < 10 84 other tunable parameters may include the... suffix length, peak-to-average-power-ratio (PAPR) parameters, and so on More examples of underwater communication systems can be found in [10], [6] and [11] 1.4 Explore or Exploit? We have two choices at every transmission: 1 Try out new link parameter values, thus exploring the parameter search space 2 Exploit the parameter values with the highest observed data rate Exploration contributes to the knowledge... small Due to the multi-path propagation and ambient noise, the effective data rates are lower and packet loss rate is usually much greater Comprehensive reviews of underwater acoustic communications are presented in [3],[4],[5] As James Preisig [6] pointed out “There is no ‘typical’ underwater acoustic environment, and no ‘typical’ underwater acoustic communications channel exists” 1 Chapter 1 Introduction... code-packet length combination j di Uncoded Link Rate of link scheme i E chij m × n channel PER matrix xfer fu Amount of data transferred successfully by burst u γj Packet efficiency factor for code-packet length combination j L All available packet lengths lj Packet length of code-packet length combination j M Maximum length of a coded packet supported by the modem m Number of link schemes n Number of code-packet... in minimum time The objective for tuning the parameters could be to optimize data rates, protect against errors, minimize power, and so on If the physics of the channel is completely known, it is possible to determine the values of these parameters for a given objective However in practice, it is quite difficult to know the state of the channel completely The parameters often interact with each other,... channel model addressed the specific parameters and constraints brought by the underwater acoustic channel This chapter formulates the problem and discusses how the understanding from the channel model can be used in finding solutions to the problem [9] This background holds significant importance in searching for various strategies for tuning of underwater acoustic parameters As discussed in the introduction, . Adapting Underwater Physical Link Parameters Using Data Driven Algorithms D. Melani Jayasuriya BSc Eng. (Hons), University of Moratuwa A. in minimum possible time using an underwater acoustic link that can be tuned by changing physical link parameters. Assuming we have no prior knowledge about the average data rate resulting from. file size. 1.5 Thesis Contributions We present a data driven approach for tuning the physical layer parameters of a communication link to optimize data rates, assuming the channel remains static over