Robust Underdetermined Algorithm Using Heuristic-Based Gaussian Mixture Model for Blind Source Separation 143 Three other underdetermined algorithms with state of the art are tested in the following simulations to compare with the proposed algorithm. Here, the first one is named PF proposed in (Bofill & Zibulevsky, 2001), the second one is named GE proposed in (Shi et al, 2004), and the last one is our previous work which named FC proposed in (Liu et al, 2006). In order to confirm validation and robustness of these algorithms, four sparse signals recorded from real sounds are taken for the source signals whose waveforms are shown in Fig. 3 and Fig. 4. In the first BSS case, the first three source signals are mixed by a well- conditioned mixing matrix as ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ = 7071.00.8944-0.9285 7071.00.44720.3714 well A (24) It involves distinguishable mixing vectors whose normal angles are [] 5000.0 ,2952.0 ,2422.0 ~ − well μ respectively. The distribution of mixtures is plotted in Fig. 5. In the second BSS case, the four source signals are mixed by an ill-conditioned mixing matrix as ⎟ ⎟ ⎠ ⎞ ⎜ ⎜ ⎝ ⎛ − = 9285.07926.07071.07071.0 3714.06097.07071.07071.0 ill A (25) It involves undistinguishable mixing vectors whose normal angles are [] 2422.0 ,4175.0 ,5000.0- ,5000.0 ~ = ill μ respectively since the first vector and the third vector are quite close. The distribution of mixtures is plotted in Fig. 6. The parameters of compared algorithms are referenced from the original setting of their articles. For example, the grid scale is given 720 and λ is entered as 55 in PF. The improved PSO, through the experience of numerous previous experiments, are given the suitable parameters, 20=sz , 5.0 = D P , 12.0 0 = α , 3.0 1 = α and 4.0 2 = α . For generation number, 100=G is given in first case and 200 = G is given in second case. For all algorithms, the every simulation will be tested by 30 independent runs. And, the performance is evaluated by mean square error (MSE) as () ∑ = −= n i ii n MSE 1 2 ~ 1 μμ (26) where i μ ~ denotes the ith real mixing vector, and i μ denotes the ith estimation. In final, the average of MSE by 30 independent runs will be presented. An estimated set of mixing vector having a small MSE implies a excellent source separation. Frontiers in Robotics, Automation and Control 144 Fig. 3. The waveforms of source signals represented in time domain. Fig. 4. The waveforms of source signals represented in frequency domain. Robust Underdetermined Algorithm Using Heuristic-Based Gaussian Mixture Model for Blind Source Separation 145 Fig. 5. The distribution of mixtures produced by well-conditioned mixing matrix. Fig. 6. The distribution of mixtures produced by ill-conditioned mixing matrix. Frontiers in Robotics, Automation and Control 146 5.2 Results After two simulations are implemented by the involved algorithms, the compared data about estimating accuracy are presented in Table 1 and Table 2. The both tables describe the real mixing vectors, the average of estimated mixing vectors and the MSE of the four algorithms for well-conditioned case and ill-conditioned case. From these tables, it could be observed that GE algorithm’s performance is always unacceptable in all cases. PF algorithm just work acceptably in well-conditioned case, but it fail in ill-conditioned case. FC algorithm is valid in all cases, but its MSE is not better than that of proposed PSO-GMM algorithm. In order to compare the improved PSO and standard PSO, their average fitness curves are shown in Fig. 7 (well-conditioned case) and Fig. 8 (ill-conditioned case). Form both figures, it could be observed that improved version has better convergent ability on speed and depth; particularly, that in Fig. 8. Compared algorithms PF GE FC PSO-GMM 2422.0 ~ 1 = μ 1 μ 0.2497 0.2292 0.2498 0.2421 2952.0 ~ 2 −= μ 2 μ -0.2190 -0.1958 -0.2903 -0.2896 5000.0 ~ 3 = μ 3 μ 0.1627 0.1402 0.5134 0.4995 MSE 0.0399 0.0465 8.7110e-05 1.0540e-05 Table 1. Comparison of results between the four algorithms in well-conditioned BSS case. Compared algorithms PF GE FC PSO-GMM 5000.0 ~ 1 = μ 1 μ 0.5520 0.6856 0.5000 0.4998 5000.0 ~ 2 −= μ 2 μ -0.4895 -0.6469 -0.4982 -0.4929 4175.0 ~ 3 = μ 3 μ 0.5639 0.5687 0.3996 0.4176 2422.0 ~ 4 = μ 4 μ 0.7494 -0.0817 0.2426 0.2426 MSE 0.0704 0.0460 8.0060e-05 1.2662e-05 Table 2. Comparison of results between the four algorithms in ill-conditioned BSS case. Robust Underdetermined Algorithm Using Heuristic-Based Gaussian Mixture Model for Blind Source Separation 147 6. Discussion In comparing the proposed PSO-GMM with related BSS algorithms, the performance of GE algorithm is sensitive to predefined parameters. Tt exhibited a large value in MSE because of the lack of perfect initiations. Unfortunately, there is no rule or criterion that can be referred to for choosing suitable initiations. The PF algorithm is available in well- conditional case, and it does not involves any random initiation. However, the PF algorithm is not robust enough to deal with a complex problem because its settings of parameters is not for general-purpose; moreover, there are no instructions to guide a user on how to adjust them to suit other specific cases. The FC algorithm and PSO-GMM algorithm are efficient and robust enough to handle whether a general toy BSS case or an advanced BSS case. For further comparison between the both algorithms, it can be discovered that PSO method explores variant potential solutions; therefore, its accuracy is more excellent than FC algorithm. For the different PSO versions, the improved PSO exhibits a better convergent curve because it have the additional mechanism which enhances and replaces the globel best solution to rapidly drag particles toward a solution with an exact direction and distance during whole generations. Fig. 7. Fitness convergence comparison between improved PSO and standard PSO for well- conditioned BSS case. Frontiers in Robotics, Automation and Control 148 Fig. 8. Fitness convergence comparison between improved PSO and standard PSO for ill- conditioned BSS case., 7. Conclusion This study addresses on the BSS problem which involves sparse source signals, underdetermined linear mixing model. Some related algorithms have been proposed, but are only tested on toy cases. For robustness, GMM is introduced to learn the distribution of mixtures and find out the unknown mixing vectors; meantime, PSO is used to tune the parameters of GMM for expanding search range and avoiding local solutions as much as possible. Besides, a mechanism is proposed to enhance the evolution of PSO. For simulations, a simple toy case which includes distinguishable mixing matrix and a difficult case which includes close mixing vectors are designed and tested on several state of the art algorithms. Simulation results demonstrate that the proposed PSO-GMM algorithm has the best accuracy and robustness than others. Additionally, the comparison between standard PSO and improved PSO shows that improved PSO is more efficient than standard PSO. Robust Underdetermined Algorithm Using Heuristic-Based Gaussian Mixture Model for Blind Source Separation 149 8. References Amari, S.; Chen, T. P. & Cichocki, A. (1997). Stability analysis of learning algorithms for blind source separation, Neural Networks, vol. 10, issue: 8, pp. 1345-1351 Belouchrani, A.; Abed-Meraim, K.; Cardoso, J. F. & Moulines, E. (1997). A blind source separation technique using second-order statistics, IEEE Trans. on Acoustics, Speech, and Signal Processing,vol. 45, issue: 2, pp. 434-444 Bofill, P. & Zibulevsky, M. (2001). Underdetermined blind source separation using sparse representations, Signal Processing, vol. 81, pp. 2353-2362 Cichocki, A. & Unbehauen, R. (1996). Robust neural networks with on-line learning for blind identification and blind separation of sources, IEEE Trans. on circuits and systems-I: fundamental theory and applications, vol. 43 Eberhart, R. C. & Kennedy, J. (1995). A new optimizer using particle swarm theory,” Proc. of the Sixth International Symposium on Micro Machine and Human, pp. 39-43 Grady, P. O. & Pearlmutter, B. (2004). Soft-LOST: EM on a mixture of oriented lines, Proc. of ICA 2004, ser. Lecture Notes in Computer Science Granada, pp. 430-436 Gudise, V. G. & Venayagamoorthy, G. K. (2003). Comparison of particle swarm optimization and Backpropagation as Training Algorithms for Neural Networks, Proc. of IEEE Swarm Intelligence Symposium, pp. 110-117 Hedelin, P. & Skoglund, J. (2000). Vector quantization based on gaussian mixture models, IEEE Trans. on Speech and Audio Processing, vol. 8, no. 4, pp. 385-401 Herault, J. & Juten, C. (1986). Space or time adaptive signal processing by neural network models, Proc. of AIP Conf. Snowbird, UT, in Neural Networks for Computing, J. S. Denker, Ed. New York: Amer. Inst. Phys., pp. 206-211 Lee, T. W.; Girolami, M. & Sejnowski, T. J. (1999a). Independent component analysis using an extended infomax algorithm for mixed sub-gaussian and super-gaussian sources, Neural Computation, vol. 11, issue: 2, pp. 409-433 Lee,T. W.; Lewicki, M. S.; Girolami, M. & Sejnowski, T. J. (1999b). Blind source separation of more sources than mixtures using overcomplete representations, Signal Processing Letters, vol. 6, issue: 4, pp. 87-90 Li, Y. & Wang, J. (2002). Sequential blind extraction of instantaneously mixed sources, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. 50, issue: 5, pp. 997-1006 Lin, C. & Feng, Q. (2007). The standard particle swarm optimization algorithm convergence analysis and parameter selection, Proc. of the 3 th International Conference on Natural Computation, pp. 823-826 Liu, C. C.; Sun, T. Y.; Li, K. Y. & Lin, C. L. (2006). Underdetermined blind signal separation using fuzzy cluster on mixture accumulation, Proc. of the International Symposium on Intelligent Signal Processing and Communication Systems, pp. 455-458 Liu, C. C.; Sun, T. Y.; Li, K. Y.; Hsieh, S. T. & Tsai, S. J. (2007). Blind sparse source separation using cluster particle swarm optimization technique, Proc. of International Conference on Artificial Intelligence and Applications, pp. 549-217 Luengo, D.; Santamaria, I. & Vielve, L. (2005). A general solution to blind inverse problems for sparse input signals: deconvolution, equalization and source separation, Neurocomputing, vol. 69, pp. 198-215 Frontiers in Robotics, Automation and Control 150 Nikseresht, A. & Gelgon, M. (2008). Gossip-based computation of a gaussian mixture model for distributed multimedia indexing, IEEE Trans. on Multimedia, vol. 10, no. 3, pp. 385-392 Pajunen, P. (1998). Blind source separation using algorithmic information theory, Neurocomputing, vol. 22, issue: 1-3, pp. 35-48 Pham, D. T. & Vrins, F. (2005). Local minima of information-theoretic criteria in blind source separation,” IEEE Signal Processing Letters, vol. 12, issue: 11, pp. 788-791 Shi, Z.; Tang, H.; Liu, W. & Tang, Y. (2004). Blind source separation of more sources than mixtures using generalized exponential mixture models, Neurocomputing, vol. 61, pp. 461-469 Shi, Y. & Eberhart, R. C. (1998). A modified particle swarm optimizer, Proc. of IEEE World Congress on Computational Intelligence, pp. 69-73 Song, K.; Ding, M.; Wang, Q. & W. Liu, (2007). Blind source separation in post-nonlinear mixtures using natural gradient descent and particle swarm optimization algorithm, Proc. of the 4 th International Symposium on Neural Networks, pp. 721-730 Tangdiongga, E.; Calabretta, N.; Sommen, P. C. W. & Dorren, H. J. S. (2001). WDM monitoring technique using adaptive blind signal separation, IEEE Photonics Technology Letters, vol. 13, issue: 3, pp. 248 - 250 Todros, K. & Tabrikian, J. (2007). Blind separation of independent sources using gaussian mixture model, IEEE Trans. on Signal Processing, vol. 55, no. 7, pp. 3645-3658 Vaerenbergh, S. V. & Santamaria, I. (2006). Spectral clustering approach to underdetermined postnonlinear blind source separation of sparse sources, IEEE Trans. on Neural Networks, vol. 17, issue: 3, pp. 811-814 Yilmaz, O. & Rickard, S. (2004). Blind separation of speech mixtures via time-frequency masking, IEEE Trans. on Acoustics, Speech, and Signal Processing, vol. 52, issue: 7, pp. 1830-1847 Yue, Y. & Mao, J. (2002). Blind separation of sources based on genetic algorithm, Proc. of the 4 th World Congress on Intelligent Control and Automation, pp. 2099-2103 Zhang, Y. C. & Kassam, S. A. (2004). Robust rank-EASI algorithm for blind source separation, IEE Proceedings-Communications, vol. 151, issue: 1, pp. 15-19 9 Pattern-driven Reuse of Behavioral Specifications in Embedded Control System Design Miroslav Švéda, Ondřej Ryšavý & Radimir Vrba Brno University of Technology Czech Republic 1. Introduction Methods and approaches in systems engineering are often based on the results of empirical observations or on individual success stories. Every real-world embedded system design stems from decisions based on an application domain knowledge that includes facts about some previous design practice. Evidently, such decisions relate to system architecture components, called in this paper as application patterns, which determine not only a required system behavior but also some presupposed implementation principles. Application patterns should respect those particular solutions that were successful in previous relevant design cases. While focused on the system architecture range that covers more than software components, the application patterns look in many features like well- known software object-oriented design concepts such as reusable patterns (Coad and Yourdon, 1990), design patterns (Gamma et al., 1995), and frameworks (Johnson, 1997). By the way, there are also other related concepts such as use cases (Jacobson, 1992), architectural styles (Shaw and Garlan, 1996), or templates (Turner, 1997), which could be utilized for the purpose of this paper instead of introducing a novel notion. Nevertheless, application patterns can structure behavioral specifications and, concurrently, they can support architectural components specification reuse. Nowadays, industrial scale reusability frequently requires a knowledge-based support. Case-based reasoning (see e.g. Kolodner, 1993) can provide such a support. The method differs from other rather traditional procedures of Artificial Intelligence relying on case history: for a new problem, it strives for a similar old solution saved in a case library. Any case library serves as a knowledge base of a case-based reasoning system. The system acquires knowledge from old cases while learning can be achieved accumulating new cases. Solving a new case, the most similar old case is retrieved from the case library. The suggested solution of a new case is generated in conformity with the retrieved old case. This book chapter proposes not only how to represent a system’s formal specification as an application pattern structure of specification fragments, but also how to measure similarity of formal specifications for retrieval. In this chapter, case-based reasoning support to reuse is focused on specifications by finite-state and timed automata, or by state and timed-state Frontiers in Robotics, Automation and Control 152 sequences. The same principles can be applied for specifications by temporal and real-time logics. The following sections of this chapter introduce the principles of design reuse applied by the way of application patterns. Then, employing application patterns fitting a class of real-time embedded systems, the kernel of this contribution presents two design projects: petrol pumping station dispenser controller and multiple lift control system. Via identification of the identical or similar application patterns in both design cases, this contribution proves the possibility to reuse substantial parts of formal specifications in a relevant sub-domain of embedded systems. The last part of the paper deals with knowledge-based support for this reuse process applying case-based reasoning paradigm. The contribution provides principles of case-based reasoning support to reuse in frame of formal specification-based system design aiming at industrial applications domain. This book chapter stems from the paper (Sveda, Vrba and Rysavy, 2007) modified and extended by deploying temporal logic formulas for specifications. 2. State of the Art To reuse an application pattern, whose implementation usually consists both of software and hardware components, it means to reuse its formal specification, development of which is very expensive and, consequently, worthwhile for reuse. This paper is aimed at behavioral specifications employing state or timed-state sequences, which correspond to Kripke style semantics of linear, discrete time temporal or real-time logics, and at their closed-form descriptions by finite-state or timed automata (Alur and Henzinger, 1992). Geppert and Roessler (2001) present a reuse-driven SDL design methodology that appears closely related approach to the problem discussed in this contribution. Software design reuse belongs to highly published topics for almost 20 years, see namely Frakes and Kang (2005), but also Arora and Kulkarni (1998), Sutcliffe and Maiden (1998), Mili et al. (1997), Holzblatt et al. (1997), and Henninger (1997). Namely the state-dependent specification-based approach discussed by Zaremski et. al. (1997) and by van Lamsweerde and Wilmet (1998) inspired the application patterns handling presented in the current paper. To relate application patterns to the previously mentioned software oriented concepts more definitely, the inherited characteristics of the archetypal terminology, omitting namely their exclusive software orientation, can be restated as follows. A pattern describes a problem to be solved, a solution, and the context in which that solution works. Patterns are supposed to describe recurring solutions that have stood the test of time. Design patterns are the micro-architectural elements of frameworks. A framework which represents a generic application that allows creating different applications from an application sub-domain is an integrated set of patterns that can be reused. While each pattern describes a decision point in the development of an application, a pattern language is the organized collection of patterns for a particular application domain, and becomes an auxiliary method that guides the development process, see the pioneer work by Alexander (1977). Application patterns correspond not only to design patterns but also to frameworks while respecting multi-layer hierarchical structures. Embodying domain knowledge, application patterns deal both with requirement and implementation specifications (Shaw and Garlan, 1996). In fact, a precise characterization of the way, in which implementation specifications [...]... possibly interleaved by induced error values, see an example timed-state sequence: inp=0 inp=0 inp=1 inp=0 inp=0 (0, q1) → → (i, q1) → (i+1, q2) → → (j, q2) inp=1 (k, qb/2+1) inp=1 → → inp=1 (m, qb-1) inp=0 (m+1, qb) inp=1 inp=1 (n, qb) → → → → inp=0/IMP → (n+1, q1) i, j, k, m, n are integers representing discrete time instances in increasing order For the sake of fault-detection requirements, the incremental... case, see the following timed-state sequence: inp=0 inp=0 inp=1 inp=0 inp=0 (0, q1) → → (i, q1) → (i+1, q2) → → (j, q2) inp=1 (k, qb/2+1) inp=1 → → inp=1 (m, qb-1) inp=0 (m+1, qb) inp=1 inp=1 (n, qb) → → → → inp=0/IMP → (n+1, q1) i, j, k, m, n are integers representing discrete time instances in increasing order The information about a detected impulse is sent to the counting automaton that can... EUROCAST'99, Springer-Verlag, LNCS 1798, 80-89 Sveda, M and R Vrba (20 06) Fault Maintenance in Embedded Systems Applications Proceedings of the Engineering of Computer-Based Systems Proceedings of the Third International Conference on Informatics in Control, Automation and Robotics (ICINCO 20 06) , INSTICC, Setúbal, Portugal, 183-1 86 Sveda, M., R Vrba and O Rysavy (2007) Pattern-Driven Reuse of Embedded Control. .. developed by refining the higher level state "position_indication" into three communicating lower level automata: two noise-tolerant impulse detectors and one checking reversible counter Frontiers in Robotics, Automation and Control 158 5.2 Incremental measurement for position and speed control Intuitively, the first automaton models the noise-tolerant impulse detector in the same manner as in previous... Design Behavioral and Architectural Specifications in Embedded Control System Designs Proceedings of Fourth International Conference on Informatics in Control, Automation and Robotics (ICINCO 2007), INSTICC, Angers, FR, pp 244-248 Turner, K.J (1997) Relating Architecture and Specification Computer Networks and ISDN Systems, 29(4), 437-4 56 van Lamsweerde, A and L Willemet (1998) Inferring Declarative... Engineering, 24(12), 1089-1114 Xinyao, Y., W Ji, Z Chaochen and P.K Pandya (1994) Formal Design of Hybrid Systems In: (Langmaack, H., W.P de Roever and J Vytopil) Formal Techniques in Real-Time and Fault-Tolerant Systems, Springer-Verlag, LNCS 863 , 738-755 Zaremski, A.M and J.M Wing (1997) Specification Matching of Software Components ACM Trans on Software Engineering and Methodology, 6( 4), 333- 369 ... measurements of the same information They provide increased reliability and accuracy Because competitive sensors are redundant, inconsistencies may arise between sensor readings, and care must be taken to combine the data in a way that removes the uncertainties When done properly, this kind of data fusion increases the robustness of the system  168 Frontiers in Robotics, Automation and Control Cooperative... Alur, R and T.A Henzinger (1992) Logics and Models of Real Time: A Survey In: (de Bakker, J.W., et al.) Real-Time: Theory in Practice Springer-Verlag, LNCS 60 0, 741 06 Arora, A and S.S Kulkarni (1998) Component Based Design of Multitolerant Systems IEEE Transactions on Software Engineering, 24(1), 63 -78 Atkinson, S (1998) Modeling Formal Integrated Component Retrieval Proceedings of the Fifth International... 42  164 Frontiers in Robotics, Automation and Control Kolodner, J (1993) Case-based Reasoning, Morgan Kaufmann, San Mateo, CA, USA Lamport, L (1994) Temporal Logic of Actions ACM Transactions on Programming Languages and Systems, 16( 3) :872-923 Lamport, L (2002) Specifying Systems Addison-Wesley, 2002 Mili, R., A Mili, and R.T Mittermeir (1997) Storing and Retrieving Software Components: A Refinement... California, 2 06- 215 Sutcliffe, A and N Maiden (1998) The Domain Theory for Requirements Engineering IEEE Transactions on Software Engineering, 24(3), 174-1 96 Sveda, M (19 96) Embedded System Design: A Case Study IEEE Proc of International Conference and Workshop ECBS' 96, IEEE Computer Society, Los Alamitos, California, 260 - 267 Sveda, M., O Babka and J Freeburn (1997) Knowledge Preserving Development: . solution to blind inverse problems for sparse input signals: deconvolution, equalization and source separation, Neurocomputing, vol. 69 , pp. 198-215 Frontiers in Robotics, Automation and Control. retrieval. In this chapter, case-based reasoning support to reuse is focused on specifications by finite-state and timed automata, or by state and timed-state Frontiers in Robotics, Automation and Control. → inp=1 → (m, q b-1 ) inp=0 → (m+1, q b ) inp=1 → inp=1 → (n, q b ) inp=0/IMP → (n+1, q 1 ) i, j, k, m, n are integers representing discrete time instances in increasing order.