... accurate word n-gram language model directly from characters of arbitrary language, without any “word” indications.1 Introduction“Word” is no trivial concept in many languages.Asian languages such ... by the character HPYLMaccording to (4).This language model, which we call Nested Pitman-Yor Language Model (NPYLM) hereafter,is the hierarchical language model shown in Fig-ure 2, where the ... describeshow to accomplish this by constructing a nested HPYLM of words and characters, with the associ-ated inference algorithm.3 Nested Pitman-Yor Language ModelThus far we have assumed that...
... a pre-liminary machine translation reranking experiment.2 Treelet Language Modeling The common denominator of most n-gram language models is that they assign probabilities roughly ac-cording ... top-down parsing and language modeling. Computational Linguistics.Ming Tan, Wenli Zhou, Lei Zheng, and Shaojun Wang.2011. A large scale distributed syntactic, semanticand lexical language model ... dependencies for structured language modeling. In Proceedings of the Associationfor Computational Linguistics. Association for Com-putational Linguistics.Ying Zhang. 2009. Structured language models for...
... before, theuser can take a break and then the system continueswith the next epoch.3 LanguageModeling Language modeling is important for many text pro-cessing applications, e.g., speech recognition ... and M. Fried-Oken.2010. Scanning methods and languagemodeling forbinary switch typing. In Proceedings of the NAACLHLT 2010 Workshop on Speech and Language Pro-cessing for Assistive Technologies, ... confusing. Finally, language model integrationwith RSVP is relatively straightforward, as we shalldemonstrate. See Roark et al. (2010) for methodsintegrating languagemodeling into grid scanning.2...
... Computer Processing ofOriental Languages, 12(1):99–121.Jianfeng Gao, Joshua Goodman, Mingjing Li, and Kai-Fu Lee. 2002. Toward a unified approach to statis-tical languagemodeling for Chinese. ACM ... LVCSR. Comp. SpeechLang., 22(4):360–373.Ronald Rosenfeld. 2000. Two decades of statistical language modeling: Where do we go from here?Proceeding of IEEE, 88(8):1270–1278.George Saon and Mukund ... andLin-Shan Lee. 1998. Statistics-based segment pat-tern lexicon: A new direction for chinese language modeling. In ICASSP, pages 169–172.7634.4 Application: Character-based SpokenDocument Indexing...
... Jelinek. 2000. Structured language modeling. Computer Speech and Language, 14(4):283–332.Ciprian Chelba. 2000. Exploiting Syntactic Structure for Nat-ural Language Modeling. Ph.D. thesis, The ... stringw alone.2In this sense, the scope of this paper islimited to the languagemodeling problem. As oneexample, the languagemodeling features might takeinto account n-grams, for example through ... Discriminative languagemodeling with condi-tional random fields and the perceptron algorithm. In Proc.ACL.Brian Roark, Murat Saraclar, and Michael Collins. 2005. Dis-criminative n-gram language modeling. ...
... Language, 8:1–38.B. Roark, M. Saraclar, and M. Collins. 2007. Discrimi-native n-gram Language Modeling. Computer, Speechand Language, 21:373–392.R. Rosenfeld. 1994. Adaptive Statistical Language ... Entropy Approach toAdaptive Statistical Language Modeling. Computer,Speech and Language, 10:187–228.A. Stolcke. 2002. SRILM – An Extensible Language Modeling Toolkit. In Proceedings of the ... HierarchicalDistributed Language Model. In Advances in NeuralInformation Processing Systems 21.H. Ney, U. Essen, and R. Kneser. 1994. On Structur-ing Probabilistic Dependences in Stochastic Language Modeling. ...
... for Statistical Language Modeling via Substructure Sharing and UptrainingAriya Rastrow, Mark Dredze, Sanjeev KhudanpurHuman Language Technology Center of ExcellenceCenter for Language and Speech ... tech-niques for exploiting syntactic, semantic and colloca-tional dependencies in language modeling. ComputerSpeech and Language, pages 355–372.S. K¨ubler, R. McDonald, and J. Nivre. 2009. Depen-dency ... ICASSP.Brian Roark, Murat Saraclar, and Michael Collins. 2007.Discriminative n-gram language modeling. ComputerSpeech & Language, 21(2).K. Sagae and A. Lavie. 2006. A best-first probabilis-tic shift-reduce...
... relationship, much like traditional language models, using con-ditional probability distributions. Unlike tradi-tional language models, however, our grounded language models condition the probability ... for improving speech recognition in video. The method uses grounded language modeling, an extension of tradition lan-guage modeling in which the probability of a word is conditioned not only ... work, we will examine the ability of grounded language models to improve perform-ance for other natural language tasks that exploit text based language models, such as Machine Translation....
... dependency and predictive clustering for language modeling. In EMNLP 2002. Gao. J., Yu, H., Yuan, W., and Xu, P. 2005. Minimum sample risk methods for language modeling. In HLT/EMNLP 2005. Osborne, ... Statistics, 9(2): 319-337. Roark, Brian, Murat Saraclar and Michael Collins. 2004. Corrective languagemodeling for large vo-cabulary ASR with the perceptron algorithm. In ICASSP 2004. Schapire, ... July 2006.c2006 Association for Computational LinguisticsApproximation Lasso Methods for LanguageModeling Jianfeng Gao Microsoft Research One Microsoft Way Redmond WA 98052 USA jfgao@microsoft.com...
... efforts have been made to utilize linguistic structure in language modeling, which for practical reasons is still dominated by trigram-based language models. There are two major obstacles to successfully ... structure acquisition. One basic approach to using linguistic structure for languagemodeling is to extend the conventional language model P(W) to P(W, T), where T is a parse tree of W. The extended ... headwords in phrases headed by compounds. Unsupervised Learning of Dependency Structure for LanguageModeling Jianfeng Gao Microsoft Research, Asia 49 Zhichun Road, Haidian District Beijing...
... size is approximately 120k words. 3 Optimizing Document Clustering for LanguageModeling For the purpose of language modeling, the topic la- bels assigned to a document or segment of a doc- ... contribution 172 Dynamic Nonlocal LanguageModeling via Hierarchical Topic-Based Adaptation Radu Florian and David Yarowsky Computer Science Department and Center for Language and Speech Processing, ... smooth language models are computed for each node n different than the root by three-way interpolating between the m-gram language model in the parent parent(n), the (m - 1)-gram smooth language...
... 2000. Statisti-cal languagemodeling with a class-based n-multigrammodel. Computer Speech & Language, 14(3):261–279.Pierre Dupont and Ronald Rosenfeld. 1997. Latticebased language models. ... classes in language modeling Hinrich Sch¨utzeInstitute for NLPUniversity of StuttgartGermanyAbstractBuilding on earlier work that integrates dif-ferent factors in language modeling, we ... Introduction Language models, probability distributions overstrings of words, are fundamental to many ap-plications in natural language processing. Themain challenge in languagemodeling is...
... language model is utilized to capture the term dependence. A language model is a probability distribution that captures the statistical regulari-ties of natural language use. In a language ... co-occurrence with the languagemodeling framework for IR. In our approach, we propose bigram and biterm models to capture the term dependence in centroid vector. Applying languagemodeling for the ... method using language model for passage retrieval for the factoid QA. They trained two language models, in which one was the question-topic language model and the other was passage language model....