斯坦福自然语言组的NLP及计算语言学的资料汇总

原文地址：http://www-nlp.stanford.edu/links/statnlp.html

Statistical natural language processingand corpus-based computationallinguistics: An annotated list of resources

Contents

Tools: Machine Translation, POS Taggers, NP chunking, Sequence models, Parsers, Semantic Parsers/SRL, NER, Coreference, Language models, Concordances, Summarization, Other

Corpora: Large collections, Particular languages, Treebanks, Discourse, WSD, Literature, Acquisition

SGML/XML

Dictionaries

Lexical/morphological resources

Courses, Syllabi, and other Educational Resources

Mailing lists

Other stuff on the Web: General, IR, IE/Wrappers, People, Societies

Tools

Machine Translation systems

Instructions

Building a baseline statistical phrase MT system

Wonderful pages about how to download a bunch of tools and some dataand put themtogether to build a very competent baseline statistical MT system: NAACL 2006WMT or 2009 WMT.

Freely downloadable

Moses

The most-used open-sourcephrase-based MT decoder. By Philip Koehn and many others.

Phrasal

A Java phrase-based MT decoder, largely compatible with the core of Moses,with extra functionality for defining feature-rich ML models. By Daniel Cer, Michel Galley, Spence Green, and others.

Joshua

A Java hierarchical MT decoder, largely based on the design of Hiero.By Chris Callison-Burch and others.

Jane

A phrase-based MT decoder by the U. Aachen group.

cdec

A primarily SCFG-based MT decoder by Chris Dyer and many others. C++.

EGYPT system

System from 1999 JHU workshop. Mainly of historical interest.

GIZA++ and mkcls

Franz Och. C++. GPL. Still often used for word alignment.

Thot

Phrase-based model building kit

Phramer

An Open-Source Java Statistical Phrase-Based MT Decoder

Syntax Augmented MachineTranslation via Chart Parsing

Andreas Zollmann and Ashish Venugopal

Free, but getting them requires hassle

Pharaohdecoder

Philip Koehn, ISI.

MTTK

Machine Translation Tool Kit. Deng and Byrne.

Part of Speech Taggers

Freely downloadable

Stanford POStagger

Loglinear tagger in Java (by Kristina Toutanova)

hunpos

An HMM tagger with models available for English and Hungarian. Areimplementation of TnT (see below) in OCaml.pre-compiled models. Runs on Linux, Mac OS X, and Windows.

MBT: Memory-based Tagger

Based on TiMBL

TreeTagger

A decision tree based tagger from the University of Stuttgart(Helmut Scmid). It'slanguage independent, but comes complete with parameter files forEnglish, German, Italian, Dutch, French, Old French, Spanish, Bulgarian,and Russian. (Linux, Sparc-Solaris, Windows, and Mac OS X versions.Binary distribution only.) Page has links to sites where you can run it online.

SVMTool

POS Tagger based on SVMs (uses SVMlight). LGPL.

ACOPOST (formerlyICOPOST)

Open source C taggers originally written by by Ingo Schröder.Implements maximum entropy, HMM trigram, andtransformation-based learning. C source available under GNU public license.

MXPOST: Adwait Ratnaparkhi's Maximum Entropy part of speech tagger

Java POS tagger. A sentenceboundary detector (MXTERMINATOR) is also included. Original version wasonly JDK1.1; later version worked with JDK1.3+. Class files, not source.

fnTBL

A fast and flexible implementation of Transformation-Based Learning in C++. Includes a POS tagger, but also NP chunkingand general chunking models.

mu-TBL

An implementation of a Transformation-based Learner (a la Brill),usable for POS tagging and other things by Torbjörn Lager. Webdemo also available. Prolog.

YamCha

SVM-based NP-chunker, also usable for POS tagging, NER, etc. C/C++open source. Won CoNLL 2000 shared task. (Less automatic than a specialized POStagger for an end user.)

QTAGPart of speech tagger

An HMM-based Java POS tagger from Birmingham U. (Oliver Mason).English and German parameter files. [Java class files, not source.]

The TOSCA/LOB tagger.

Currently available for MS-DOS only. But the decision to make thisfamous system available is very interesting from an historicalperspective, and for software sharing in academia more generally.LOB tag set.

The venerable Brill's Transformation-based learning Tagger

A symbolic tagger, written in C. It's no longer available from acanonical location, but you might find a version from the Wikipedia page or you could try a reimplementation suchas fnTBL.

Original Xerox Tagger

A common lisp HMM tagger available by ftp.

Lingua-EN-Tagger

Perl POS tagger by Maciej Ceglowski and Aaron Coburn. Version0.11. (A bigram HMM tagger.)

Free, but require registration

TATOO

The ISSCO tagger. HMM tagger. Need to register to download.

PoSTech Koreanmorphological analyzer and tagger

Online registration.

TnT - A StatisticalPart-of-Speech Tagger

Trainable for various languages, comes with English and Germanpre-compiled models. Runs on Solaris and Linux.

Usable by email or on the web, but not distributed freely

Memory-based tagger

From ILK group, Catholic University Brabant (Jakub Zavrel/WalterDaelemans). Does Dutch, English, Spanish, Swedish, Slovene. Other MBLdemos are also available.

Birmingham tagger

Accepts only plain ASCII email message contents. The tagset used is similar to the Brown/LOB/Penn set.

CLAWS tagger

The UCREL CLAWS tagger is available for trial use on the web. (It'slimited to 300 words though -- this site is more of an advertisement forlicensing the real thing -- available as software for Suns or as a paidservice.) You can also find info on CLAWS tagsets,though that page doesn't seem to link to the C7 tagset.

TheAMALGAM tagger

The AMALGAMProject also has various other useful resources, in particular a webguide to different tag sets in common use. The tagging is actuallydone by a (retrained) version of the Brill tagger (q.v.).

XeroxXRCE MLTT Part Of Speech Taggers

Tags any of 14 languages (European and Arabic), online on the web.

Portuguese taggers on the web: Projecto Natura and a QTAG adaptation.

Not free

Lingsoft

Lingsoft in Finland has (symbolic)analysis tools for many European languages. More information can beobtained by emailing [email protected]. Thereis an online demo.

Conexor

Conexor in Finland hasdemonstrations of EngCG-style taggers and parsers, for English, Swedish,and Spanish.

Xerox

Xerox hasmorphological analyzers and taggers for many languages.There are demos of some of their tools on the web.More information can beobtained by contacting Daniella Russo.

Infogistics

Infogistics, anEdinburgh spinoff has a tagging and NP/Verb group chunkeravailable commercially, including an evaluation version.

No longer available

LT POS and LT TTT

The Edinburgh Language Technology Group tagger and text tokenizer (andsentence splitter were binary-only Solaris tools which no longer seem tobe available.

NP chunking

Downloadable

YamCha

SVM-based NP-chunker, also usable for POS tagging, NER, etc. C/C++open source. WonCoNLL 2000 shared task. (Less automatic than a specialized POStagger for an end user.)

MarkGreenwood's Noun Phrase Chunker

A Java reimplementation of Ramshaw and Marcus (1995).

fnTBL

A fast and flexible implementation of Transformation-Based Learning in C++. Includes a POS tagger, but also NP chunkingand general chunking models.

Generic sequence models

Downloadable

CRF++

Generic CRF-based model in C++. Open source. By the author of YamCha.

Carafe

Generic CRF-based sequence models in O-CaML. Open source. By BenWellner.

FreeLing

A largesuite of language analyzers. Written in C++.Covers text preprocessing, morphology, NER, POS tagging, parsing.

Parsers

Information on available probabilistic parsers can be found on theFSNLP: probabilistic parsing links page.

Semantic Parsers

Downloadable

ASSERT

PropBank semantic roles (and opinions, etc.) by Sameer Pradhan.

Shalmaneser

FrameNet-based by Katrin Erk.

TreeKernels in SVMlight by Alessandro Moschitti.

A general package, but ithas particularly been used for SRL.

Named Entity Recognition

Downloadable

Stanford NamedEntity Recognizer

A Java Conditional Random Field sequence model with trained modelsfor Named Entity Recognition. Java. GPL. By Jenny Finkel.

LingPipe

Tools include statistical named-entity recognition, a heuristic sentenceboundary detector, and a heuristic within-document coreferenceresolution engine. Java. GPL. By Bob Carpenter, Breck Baldwin and co.

YamCha

SVM-based NP-chunker, also usable for POS tagging, NER, etc. C/C++open source. WonCoNLL 2000 shared task. (Less automatic than a specialized POStagger for an end user.)

Coreference (Anaphora) Resolution

Downloadable

BART

A Beautiful Anaphora Resolution Toolkit. Java. By YannickVersley and many others. Java. Apache with GPL components.

Guitar

Java. GPL.

Language modeling toolkits

Downloadable

IRSTLM ToolkitCompatible with SRILM, suitable for very large language models. LGPL.By Marcello Federico, Nicola Bertoldi et al.

CMU-CambridgeStatistical Language Modeling toolkit

Downloadable, but requires registration

The SRI LanguageModeling toolkit

by Andreas Stolcke is another good system forbuilding language models, freely available for research purposes.

Not yet classified

Lextools

A package of tools for creating weighted finite-statetransducers (WFST) from high-level linguistic descriptions.Lextools binaries are available free for non-commercial useat: http://www.research.att.com/sw/tools/lextools/.Supported platforms are: linux (i686), sgi (mips2) and sun4.Lextools is built on top of, and requires, the AT&T WFSTtoolkit (version 3.6), available free for non-commercial usefrom: http://www.research.att.com/sw/tools/fsm/

Friendly concordancing and text analysis tools

Wordsmith Tools (Mike Scott)

The thing to get if you are working in the Windows world.

Text summarization tools

A prototype JavaSummarisation applet (System Quirk)

MEAD

A public domain portable multi-document summarizationsystem. (Dragomir Radev and others.)

Other

Downloadable

Tilburg University's TiMBL

Tilburg's Memory Based Learner by Walter Daelemans et al. A generalnear-neighbour-based machine learning package, but optimized for statistical NLPapplications.

splitta

Statistical sentence boundary detection by Dan Gillick.

TimeExpression taggers

TIMEX2 standard taggers (site at Mitre).

NLTK

An open source Python package for NLP application development withtools such as tokenization, POS TAGGING and parsers by Ed Loper and Steven Bird.

Ted Pedersen's code

Ngram Statistics Package: Perl code that implements: Fisher's exact test, thelikelihood ratio, Pearson's chi squared test, the Dice Coefficient, and Mutual Information; Duluth Senseval-2 wordsense disambiguation systems; Senseval-1 data in Senseval-2format; various other WSD datasets in Senseval formats, andsemantic distances derived via WordNet.

ISIPtools

The main aim is a publically available speech recognitionsystem (alpha release available), but along the way there are alsotoolkits for discrete HMMs and statistical decision trees, andfor various aspects of signal processing.

Mem. A Perlimplementation of Generalized and Improved Iterative Scaling

by Hugo WL ter Doest.

Automorphology

A system (for Windows) for automatically learning the morphologicalforms of words in a corpus by John Goldsmith.

Wordnet

Wordnet is available by ftp,compiled for a variety of machine types. For money, one can also get EuroWordNet for variousEuropean languages, an Italian/English/Spanish MultiWordNetand there's now a site for Global Wordnet.(See also Mappings between WordNet versions and Perl WordNet-Similarity module by Ted Pedersen, and WordNet Domains (coarse-grained sense topic classifications).)

Penn XTAG project

A wide-coverage tree-adjoining grammar written in a mixture of Cand Common Lisp. Also includes a large coverage morphologicalanalyzer. Now includes more tools such as TCL/Tk tree viewer.

Dan Melamed'sAssorted Tools

A collection of various tools including a simulated annealling program, apost-processor for English stemming for the Penn XTAG morphologysystem, Good-Turing smoothing software, general text processing tools,text statistics tools and bitext geometry tools (mainly written in Perl 5).

MULTEXT

Constructing corpora and tools for processing multilingual corpora.Contact: Jean Veronis [email protected]. Some stuffincluding a multilingual text editor is downloadable. MULTEXT EAST has parallel versionsof Orwell's 1984 available free (upon registration) for a numberof Central European languages.

NaiveBayes algorithm

Software from the Rainbow/Libbow software package that implementsseveral algorithms for text categorization, including naive Bayes,TF.IDF, and probabilistic algorithms. Accompanies Tom Mitchell's ML text.

HDDI

Text Data Mining API from Lehigh University.

Emdros: a text database engine for linguistic analysis and research

Chasen

Japanese morphological analyzer. Descendent of JUMAN.

Free, but require registration

Stuttgart's IMSCorpus Workbench (CWB)

A workbench for full-text retrieval from large corpora (with a query language and corpus indexing). Includes the Corpus Query Processor (CQP) and xkwic.Available free for research groups (currently only as Solaris 1/2 or Linux binaries), on signing a license agreement.

Gate

University of Sheffield's General Architecture for Text Engineering. Primarily an Information Extraction system.

MITRE's Alembic Workbench

A workbench for the development of tagged corpora. Includes atagger based on Brill's TBL approach.

SNoW

SNoW is a learning program that can be used as a general purpose multi-class classifier and is specifically tailored for learning in the presence of a very large number of features. The learning architecture is a sparse network of linear units over a pre-defined or incrementally acquired feature space (Dan Roth).

Unsure

INTEX

a finite-state transducer analysis system for English, French, andItalian that runs under NextStep. Contact:Max Silberztein [email protected]

The PennToolspage collects information on a variety of NLP systems, many of which areavailable externally.

Corpora

Large collections aimed at the NLP community

LDC (LinguisticData Consortium) and its catalogue by year.

Email: [email protected]. Provides the largest range ofcorpora on CD-ROM. Cost ranges from cheap (e.g., ACL-DCI disk) to pricey.CDs can be purchased individually; institutions can become members andreceive discounts on CDs. There's an LDC Online service forsearches over the web (mainly intended for members, but there are samplersavailable).

European LanguageResources Association and its catalogue.

Distribution agency is ELDA.Rapidly growing collection of materials in European languages.

ICAME(International Computer Archive of Modern English)

Sells various corpora (includingBrown and London-Lund). Information on corpora on the web, by sending themessage help to [email protected], by ftp to nora.hd.uib.no.Also, manuals forthese corpora.

Reuters @NIST

Reuters corpora are now distributed by NIST.

TRACTOR

TELRI Research Archive of Computational Tools and Resource.Corpora, many multilingual, in European community languages. Small feefor joining in order to be able to get corpora (unless you havecontributed corpora).

CLR (Consortium for LexicalResearch)

Email: [email protected]. Focuses more on languageprocessing tools and lexicons, but does have some corpora. As of Feb 1996,you can get most of their stuff by anonymous ftp to clr.nmsu.edu. Their catalog isavailable as a postscript file.

OTA (Oxford Text Archive)

Provides mainly literary texts. Has a bright new website. Email: [email protected].Most materials are available on the web or by anonymous ftp to ota.ox.ac.uk.Some require negotiations with the providers.

Leipzig Corpora Collection

Sentence collections in MySQL database for 17 mainly European languages.

BNC (British National Corpus)

A 100 million word corpus of British English. Youcan search it online from their simple webinterface or via View, a muchbetter interface by Mark Davies, and there is an index togenres by David Lee. And now, an XML edition.

European CorpusInitiative Multilingual Corpus I (ECI/MCI)

A 98 million word corpus, covering most of the majorEuropean languages, as well as Turkish, Japanese, Russian, Chinese, andMalay. Cheap. Need to sign a license agreement available at either theWWW site. Also available from the LDC.

Survey of English Usage

At the Department of English Language andLiterature at University College London. Includes the British part ofICE, the InternationalCorpus of English project. Now availabletagged, and parsed for function. 83,419 sentences. Includes ICECUP,dedicated retrieval software. Also, DiachronicCorpus of Present-Day Spoken English (800,000 words, tagged andparsed, half from ICE-GB and half from London-Lund).

International Corpus of English (ICE)

Million word collections of English from various world Englishes: ICE-NZ,ICE-HK, ICE-East Africa, etc. Severalof them are downloadable from this site.

Corporaheld by Lancaster University

This link provides its own annotations.

The European LanguageActivity Network

Promises a uniform query language for accessing corpora in all EUlanguages -- but isn't quite there yet.

Talkbank.

Rich video and transcripts.

Particular languages

English

English language corpora available from the sites above are not repeatedhere.

Corpora by Geoffrey Sampson's team

The SUSANNE corpusand the CHRISTINEcorpus (SUSANNE markup of a speech corpus).

Michigan Corpus of Academic Spoken English (MICASE).1.7 million words from 1997-2001.

Penn-Helsinki Parsed Corpus ofMiddle English

A syntactically annotated corpus of the Middle English prosesamples in the Helsinki Corpus of Historical English, withadditions. 1.3 million words. $200.

Corpus of Professional, SpokenAmerican-English (CPSA)

2 million words from faculty and committee meetings and White Housepress conferences (50K work sample free on internet).

Lancaster Parsed Corpus

Dialogue DiversityCorpus (Bill Mann)

American NationalCorpus

Chinese

English language corpora available from the sites above are not repeatedhere.

The Lancaster Corpus of Mandarin Chinese (LCMC)

By Tony McEnery and Richard Xiao. Distinguished by being a balancedcorpus, and freely available.

Multilingual

JRC-Acquis

A parallel corpus of EU documents across all member states. 8 million words or more in each of 20 languages.

EMILLE/CIIL

Monolingual written corpus data for 14 SouthAsian languages (Assamese, Bengali, Gujarati, Hindi, Kannada, Kashmiri,Malayalam, Marathi, Oriya, Punjabi, Sinhala, Tamil, Telegu and Urdu).Orthographically transcribed spoken data and parallelcorpus data for five South Asian languages (Bengali, Gujarati, Hindi,Punjabi and Urdu). In addition, the parallel corpus contains the Englishoriginals from which the translations stored in the corpus were derived.All data in the corpus is CES and Unicode compliant. The EMILLE corpustotals some 94 million words. Downloadable.

OPUS

An open source parallel corpus, aligned, in many languages, based onfree Linux etc. manuals.

WorldHealth Organization Computer Assisted Translation page.

Also includes a good selection of links on Computer AssistedTranslation. (See also thecopyright page.)

SearchableCanadian Hansard French-English parallel texts (1986-1993)

From the Laboratoirede Recherche Appliquée en Linguistique Informatique,Universite de Montréal

European Union web server

Parallel text in all EU languages. (In particular try European legislation.)

TELRI CD-ROMs

Parallel and other text in central and eastern european languages.

Bosnian

The Oslo Corpusof Bosnian Texts.

Czech

Parallel Czech-English

Literature translations in Czech and English

Czech National Corpus project: SYN2000

100 million words of contemporary Czech.

French

Association des BibliophilesUniversels

Various French literary works.

American andFrench Research on the Treasury of the French Language (ARTFL)

150 million word corpus of various genres of French. You have to be amember to use it (but membership is fairly cheap).

German

COSMASCorpus

Large (over a billion words!) online-searchable German and Austriancorpora. This is the publically available part of the 1.85billion word Mannheimer Corpus Collection

NEGRACorpus

Saarland University Syntactically Annotated Corpus of GermanNewspaper Texts. Available free of charge to academics. 20,000sentences, tagged, and with syntactic structures. Free for academic use.

Russian

Russian National Corpus

150 million words, 5 million words POS-tagged, some in dependencytreebank.

Library ofRussian Internet Libraries

Various literary works.

Slovene

Slovene-English parallel corpus

1 M words, free to download + on-line concordances.

Coming soon: Slovene referencecorpus of 100 M words

Croatian

Croatian National Corpus

100 M words

Spanish and Portuguese

TychoBrahe Parsed Corpus of Historical Portuguese

Over a million words ofPortuguese from different historical periods, some of itmorphologically analyzed/tagged. Free.

Information about MarkDavies' collection of (mainly historical Spanish and Portuguese.

It's not clear what their availability is.

The CUMBRE corpus. Contact ProfessorAquilino Sánchez

The CRATER Spanish corpus

Morphosyntactically tagged telecommunicationmanuals) is available by ftp.

Corpusresources for Portuguese

In total about 70 million words, available free, from varioussources (newswire, etc.)

Folha de S. Paulo newspaper

4 annual CDROMs with full text.

COMPARA

Portuguese-English parallel corpus. (In general, various resourcesat Linguateca site.

See also under ELRA, above.

Swedish

Spraakdata, Departmentof Swedish, Göteborgs University.

Has various searcable part of speechtagged Swedish corpora (Parole, Bank of Swedish, etc.), and somematerial in Zimbabwean languages.

Treebanks

Name	Language	Size	Availability	Comments
Penn Treebank	US English	2 million + words	Available (distributed by LDC)	1 million WSJ, 1 million speech, surface syntax (1970s TG)
BLLIP WSJ corpus	US English	30 million words	Available (distributed by LDC)	WSJ newswire. Automatically parsed, not hand checked. Same structure as Penn Treebank, except for some additional coreference marking
ICE-GB	UK English	1 million words (83,394 sentences)	Available; c. 500 pounds	British part of ICE, the International Corpus of English project. Tagged and parsedfor function. Half spoken material.
Bulgarian Treebank	Bulgarian	n/a	POS-tagged texts and dependencies analyses are available (some are free on the web, others via a license agreement)	An under construction Bulgarian HPSG treebank.
Penn Chinese Treebank	Chinese	100,000 words	Available (LDC)	Based on Xinhua news articles. 1980s-style GB syntax.
The Prague DependencyTreebank 1.0	Czech	500,000 words	Free on completion of license agreement (available through LDC).	Analyzed at thelevels of parts of speech, syntactic functions (and, in the future,semantic roles) level in a dependency framework. Text from newspapers and weekly magazines.
Danish Dependency Treebank 1.0	Danish	100,000 words	Available free under the GPL.	Built on a portion of the Parole corpus.
Alpino Dependency Treebank	Dutch	150,000 words	Freely downloadable	Assorted subcorpora. By far the largest isthe full cdbl (newspaper) part of the Eindhoven corpus.
NEGRA Corpus	German	20,000 sentences	Available free of charge to academics on completion of license agreement.	Saarland University Syntactically Annotated Corpus of German Newspaper Texts. Tagged, and with syntactic structures.
TIGER corpus	German	700,000 words	Available free of charge for research purposes on completion of license agreement.	German newspaper text (FrankfurterRundschau). Semi-automatically parsed.They also have a good treebank search tool, TIGERSearch.
Icelandic Parsed Historical Corpus (IcePaHC)	Icelandic	1,000,000 words	Free download (LGPL)	Texts from 1150 through 2008!
TUT:Turin University Treebank	Italian	2,400 sentences	Free download.	Morhpological analysis and dependency analysis. Penn Treebank translation.Civil law and newspaper texts.
Floresta Sintá(c)tica	Portuguese	168,000 words hand-corrected; 1,000,000 words automatically parsed	Hand corrected part is free web download; automatically parsed part available through email contact	Text from CETEMPúblicocorpus. Phrase structure and dependency representations. Available in several formats, including Penn Treebank format.
Talbanken05	Swedish	300,000 words	Free download	Resurrects and modernizes an early treebank from the 1970s.

Verbmobil Tübingen: under construction treebanked corpus of German,English, and Japanese sentences from Verbmobil (appointmentscheduling) data

Syntactic Spanish Database (SDB)University of Santago de Compostela. 160,000 clauses / 1.5 million words.

CKIP Chinese Treebank (Taiwan). Based on Academia Sinica corpus.(There's also a 100sentence Chinese treebank at U. Maryland.)

LDC Korean Treebank.

Dublin-EssexTreebank project

Deriving Linguistic Resources from Treebanks.

Treebanks

CSTBank:Cross-document Structure Theory: marking sentence functionalrelationships across related documents.

Resources for Word Sense Disambiguation

The Senseval web site

Has acomprehensive selection of resources for WSD, including a good list of WSD data resources, but not yet the new SEMCOR.

Ted Pedersen's code

Includes various WSD systems.

SenseClusters

Open source package for unsupervised discovery of word senses by clusteringtogether instances of a word (or words) that are used in similar contexts in raw text, supporting a wide range of clustering techniques based on both context vectors and similarity matrices, and including links toSVDPACKC and CLUTO. Ted Pedersen and Amruta Purandare.

EvocationWordNet synset similarity judgments

Judgments on how similar the meanings of synsets are and how commonthey are in the BNC from Jordan Boyd-Graber.

Literature

There are now quite large collections of online literature, available invarious languages (though the majority are in English, of course). Beloware pointers to some of the main collections:

Entirely or mainly English

Alex: A Catalogueof Electronic Texts on the Internet

Seems to have one of the largest collection. Searching and browsingfacilities through gopher menus. Many languages.

Wiretap Electronic Text Archive

Extensive and good quality. Still in the gopher age, though.

The On-line BooksPage

The index here only covers books in English, but there are lots oflinks to other collections of material in all languages.

Project Gutenberg

The oldest and largest project to get out of copyright literatureonline, freely available. (Or see the mirror, Sailor's ProjectGutenberg site.)

The Electronic TextCenter of the University of Virginia

Large collection of SGML text, mainly in English, but also in othermajor languages.

Center for Electronic Texts in theHumanities

Princeton/Rutgers collaboration. They didn't have it together withtheir web site when I stopped by, but they may soon.

Oxford Electronic Text Library Editions

Available fromOxford University Press, 200 Madison Ave, NY, NY 10016 212-679-7300.The Complete Works of Jane Austen is $95.00, and is reviewed in Computers and the Humanities, 28:4-5 (Aug/Oct, 1994), 317-321.

Coreference annotated texts

From University of Woverhampton (R. Mitkov, C. Barbu et al.).

Acquisition data

CHILDES database.

Database of child language transcriptions in English and many otherlanguages. Texts are also available by ftp. Certainusage requirements. Manuals and programs for accessing the data (theCLAN concordancer) are also available online. Now in Unicode XML.

SGML/XML

Robin Cover's SGML/XMLWeb Page

This is a wonderful compendium of information on SGML and XML, including information onthe Text Encoding Initiative (TEI). This document is also a guide tomany text collections (ones using SGML).

Information about the Text EncodingInitiative (TEI). (The Pizza Chef acts asa TEI tag set selector.)

Xaira

XML Aware Indexing and Retrieval Application. The successor of SARA.

Microsoft's XML page

W3C XML page.

The Corpus EncodingStandard.

An SGML instance designed for language engineering applications.Also the XML version.

Dictionaries

Dictionaries of subcategorization frames

The following dictionaries all list surface subcategorization frames (eachwith a different annotation scheme). They are also all available inelectronic form from the publishers (not free).

COBUILD

Collins Cobuild English Language Dictionary. London: Collins, 1987.The COBUILD web sitelets you search their Bank of English corpus (but you need to pay to getmore than a trial.

LDOCE

Longman Dictionary of Contemporary English. Burnt Mill, Essex:Longman, 1978.

OALD

Oxford Advanced Learner's Dictionary of Current English. Oxford:Oxford University Press, Fourth Edition, 1989. The third edition also hadinformation on subcategorization frames, although in a differentincompatible format. However, a partial version ofthe third edition (with this information) is available free onlinefrom the Oxford Text Archive.

Not exactly a dictionary, but other popular sources are:

Levin (1993)

Beth Levin. 1993. English Verb Classes and Alternations: A PreliminaryInvestigation. Chicago. Discusses linguistic distinctions (likeunergative/unaccusative verbs, dative shift, etc., not made by the abovedictionaries). Theindex of verbs is online.

Englishsubcategorization evaluation resources

Gold standard data, from Cambridge University (Anna Korhonen)

See also COMLEX and CELEX available from the LDC.

Dictionaries of assorted languages on the web

The old version of RobertBeard's Web of Online Dictionaries long ago mutated into YourDictionary.com. I'm told theIPO has been delayed. Nevertheless, it's the most comprehensive indexof dictionaries available on the web.

Names

U.S. names with frequency information, are available from the Census Bureau.

SGML structured dictionaries

Cambridge International Dictionary of English and other products in SGML.

Lexical/morphological resources

EnglishSENSEVAL Resources

Dictionary entries and tagged examples for 35 words.

ARIES Natural Language Tools

Lexicons and morphological analysis for Spanish. There is a freeProlog demonstrator, but the real lexicons and C/C++ access tools cost money.

Courses, Syllabi, and other EducationalResources

"Techie"

Foundations of Statistical Natural LanguageProcessing

Some information about, and sample chapters from, ChristopherManning and Hinrich Schütze's new textbook, published in June 1999by MIT Press. Read about courses using this book.

Corpus-based Linguistics

Christopher Manning's Fall 1994 CMU course syllabus (a postscript file).

Statistical NLP: Theory and Practice

Christopher Manning's Spring 1996 CMU course materials.

John Lafferty andRoni Rosenfeld's Spring 1997 CMU course Language and Statistics.

Boston University (JohnD. Burger and Lynette Hirschman)

A good course and web site, by the looks!

Draft of Data-IntensiveLinguistics

By Chris Brew and Marc Moens.

StatisticalNatural Language Processing course

By Joakim Nivre. Elsnet suported.

ShortCourse: Statistical Methods in NLP

By Philip Resnik

Linguist's Guide toStatistics by Brigitte Krenn and Christer Samuelsson.

Statisticaland Corpora Based Methods for Processing Natural Languages

By AlonItai, Technion Computer Science Department. (Don't read those olddrafts of mine though ... get the real thing!)

CS 241 Statistical Models inNatural-Language Processing

Eugene Charniak, Brown University.

Michael Littman, Duke: 1997, 1998.

"Corpus Linguistics"

A tutorialon concordances and corpora by Cathy Ball

Tony Berber Sardinha'sCorpus Linguistics course

Powerpoint slides in an interesting mixture of English andPortuguese (plus the rest of his homepage!)

Concordancing andcorpus linguistics

Notes prepared by Phil Benson, Hong Kong University.

Computational Approaches toCollocations

Discussion of all the measures that have been used, and software forcalculating them. By Evert and Krenn.

Mailing lists

Mailing lists that have information on these topics include:

Corpora

The main mailing list for info on corpus-based linguistics. Subscribe by sending the message: subscribe corpora to [email protected]. Or if you want to subscribe with a differentemail address, send: subscribe corpora email-address (Note that you're now speaking to a Majordomo server, not a listserv, so you don't send your name!). Or you can subscribeon the web.

Empiricist

The empiricist list appears to be defunct now. You used to send a"subscribe" message to [email protected].

Other stuff on the Web

General resources

NIST Human Language Technology programs

Including: TREC, TIDES, ACE, ....

Text summarization

Tons of resources (tutorialis, bibliographies, and software) fordocument summarization, maintained by Dragomir Radev.

PropositionBank @ UPenn

Statistical MT

Bookmarks for Corpus-based LinguistsAn extensive annotated collection by David Lee, aimed at linguisticsmore than NLP (includes web-searchable corpora and concordancing options).

HLTCentral

European site aiming to increase transfer of language technologies to the commercial market. News, etc.

Linguisticannotation

A description of formats for linguistic annotation by Steven Bird.

CTITextual Studies, University of Oxford, Guide to Digital Resources

Lists text analysis tools, corpora, and other stuff.

U. Essex W3-Corpora

Lots of teaching material, links, and online corpora.

ComputationalLinguistics and NLP (Kenji Kita, Tokushima U.)

A good well organized list of CL references, concentrating oncorpus-based and statistical NLP methods. See also Softwaretools for NLP.

HLT Central

European Human Language Technology site

Survey ofthe State of the Art in Human Language Technology

ACL SIGLEX list ofLexical Resources

Onlinematerials for a course on Learning Dynamical Systems at BrownUniversity.

Lots of neat info.

Expert Advisory Groupfor Language Engineering Standards (EAGLES) home page

European standards organization.

Materials preparedfor Michael Barlow's Corpus Linguistics course

Corpus Linguistics University ofBirmingham

Chris Brew'sTeaching Materials for statistical NLP

Not much there last time I looked; you might also try his home page.

Edinburgh LTG HelpDesk's FAQ

Many of the questions in the concern issuesrelated to corpora and tagging.

Content AnalysisResources

Qualitative Text Analysis, Concordances, etc.

MT paper archive

Lots of papers, etc.

Information Retrieval

The SMART IR system

ACM SIGIR

Managing Gigabytes

TREC conference

Text-based IntelligentSystems (Bruce Croft)

Information Extraction/Wrapper Induction

Introduction toInformation Extraction Technology. A tutorial by Douglas E. Appelt and David Israel.

IE data sets

Updated versions (i.e., now well-formed XML) of classic IE data sets:Seminar Announcements and Corporate Acquisitions.

Web-> KB. CMU World Wide Knowledge Base project (Tom Mitchell). Has a lot of the best recent probabilistic model IE work, and links to data sets.

RISE: Repository ofOnline Information Sources Used in Information Extraction Tasks, including links to people, papers, and many widely used data sets, etc.(Ion Muslea). Appears to not have been updated since 1999.

MessageUnderstanding Conference (MUC) information. A US government fundedinformation extraction exercise (from the 1990s).

Web IR and IE (Einat Amitay).Various links on IR and IE on the web.

Web question answering system (University of Michigan)

GATE: General Architecture for TextEngineering (Sheffield)

Genia Project.Biomedical text information extraction corpus (Tsujii lab). And IEtutorial slides.

People's homepages

Home pages with something useful on them.

University of Texas at AustinMachine Learning Research Group

Steven Abney (until 1997)

Adam Berger

Various stuff on statistical MT and maximum entropy models

Alex ChengyuFang

Provides a lot of info on the kinds of things they get up to at UCL,without actually giving you anything to play with yourself.

Societies/Journals

International QuantitativeLinguistics Association/Journal of Quantitative Linguistics

Not very hip.

Association for ComputationalLinguistics/Computational Linguistics

Hipper