Japanese Dependency Corpus

Overview

We take sentences from various domains to allow corpus users to conduct domain adaptation experiments.

The unit of JDC is word like other languages contrary to existing Japanese corpora whose unit is phrase called bunsetsu. For the definition of word, we follow "short-unit words" of the Balanced Corpus of Contemporary Written Japanese (BCCWJ), which is a mature standard created by linguists of Japanese language. The only difference is that we separate the endings of inflectional words (adjectives, verbs, and auxiliary verbs) from their stems.

The target of our research is written Japanese, which is a head-final language. We assume that in Japanese dependencies go from left to right and that every word except for the last one in a sentence depends on exactly one other word. We do not make the assumption that dependencies do not cross, because even in written Japanese such dependencies may occur in informal contexts.

Corpus Specification

Each word, except for the root word, is annotated with its head (dependency destination). Thus the number of dependencies in a corpus is equal to the number of words minus the number of sentences.

The JDC are composed of the following sources:

• BCCWJ Core data

  The followings are subcategories of BCCWJ Core data:

  • OC: Yahoo! questions and answers
  • OW: White papers
  • OY: Yahoo! blog
  • PB: Books
  • PM: Magazines
  • PN: Newspaper articles

• Dictionary example sentences
  • EHJ: The example sentences in a dictionary for daily conversation

• Economy newspaper articles
  • NKN: Nikkei newspaper

• Recipes
  • RCP: Cookpad data

• Paper abstracts:
  • JNL

• Invention disclosures
  • NPT: A part of NTCIR-9 shared task set for patent machine translation.

Dependency annotation standard

Simple sentence

Basically Japanese is an SOV language. That is to say, the word order in a simple sentence is subject, object, and verb. Almost all noun phrases have a case marker called postposition to clarify its role to the verb. The only limitation is to put the main verb phrase at the end. That is to say, subject (subj.), direct object (d-obj.), indirect object (i-obj.), and other verb modifier such as adverbial phrases are ordered freely.

Compound word

We annotate a compound word with the structure representing its meaning. Modifiers of a compound word depend on its head (in many cases with very few exceptions which modifies a part of a compound word) and there is only one dependency arc going out from the head.

Copula

Some sentences have a copular verb. Most copula sentences fall into the following type:

N 1 は/subj. N 2 だ/is

We decided that the case marker "は/subj." depends on N 2, not on the auxiliary verb "だ/is." The reason is that an auxiliary verb can be omitted especially in a that-clause or sentence coordination. The head of the case marker is always N 2 independent from the existence of an auxiliary verb.

Coordination

In a coordination structure two or more phrases are concatenated by using a coordination marker. In Japanese the most frequent marker is "と/and." This marker is similar to "and" in English but we put one at each point between elements.

Dependency Parsing Experiments

Training corpus:

• BCCWJ: 6,406 sentences ( BCCWJ-2012-train )
• All full: 29,863 sentences ( BCCWJ-2012-train + {EHJ, NKN, RCP, JNL, NPT}-train )
• (under const.) +partial: All full + partial

Download

Links

Members

• Shinsuke Mori
• Hideki Ogura
• Tetsuro Sasada

Reference

A Japanese Word Dependency Corpus

Shinsuke Mori, Hideki Ogura, Tetsuro Sasada

LREC, pp.753-758, 2014.

A Pointwise Approach to Training Dependency Parsers from Partially Annotated Corpora

Daniel Flannery, Yusuke Miyao, Graham Neubig, Shinsuke Mori

Natural Language Processing, Vol.19, No.3, pp.167-191, September, 2012.

Training Dependency Parsers from Partially Annotated Corpora

Daniel Flannery, Yusuke Miyao, Graham Neubig, Shinsuke Mori

IJCNLP, pp.776-784, 11/10, 2011.