WIT3

Web Inventory of Transcribed and Translated Talks

Home 2012-02-plain Training and development sets for the MT track

For some language pairs, plain texts for MT experiments can be downloaded from this table. The names of languages are represented by TED codes, mostly the same as ISO 639-1 codes. Numbers refer to millions of units (untokenized words). (row,col) entries provide the size of the target side of parallel training data available for row-to-col language pair. Each entry is linked to the tar archive of the data for the corresponding language pair, just click on it for downloading. The color of the entry indicates what the archive includes: green if parallel and monolingual training sets, and development/evaluation set(s) are provided; blue if only parallel training data is there.
Notes:
 - in general, data included in an entry (L1,L2) differ from the data in the entry (L2,L1), due to the asymmetry of rebuilding sentence and text cleaning operations
 - sentences were not rebuilt in language pairs having either Chinese or Japanese as target language; in such cases, the original segmentation in subtitles from TED documents is kept


ar

bg

cs

de

el

en

es

fa

fr

he

hu

it

ja

ko

nl

pl

pt

pt-br

ro

ru

tr

vi

zh-cn
ar 2.120.882.031.372.292.261.262.441.661.172.210.241.602.001.711.282.262.241.631.511.660.32
bg1.95 0.912.091.402.442.331.272.511.711.202.290.251.652.071.751.312.332.321.681.551.710.33
cs0.901.00 1.020.901.121.070.881.160.830.731.050.150.751.050.820.831.071.080.870.761.120.16
de1.802.010.89 1.392.222.171.262.331.601.202.080.241.531.961.641.282.152.151.611.481.710.31
el1.211.330.781.39 1.501.421.041.541.080.931.400.181.021.381.101.021.431.431.151.011.360.21
en1.962.210.932.141.42 2.371.312.511.741.232.290.251.682.141.791.342.372.371.711.581.760.33
es1.952.190.922.131.402.46 1.282.541.721.212.310.251.662.101.771.322.352.341.701.561.730.33
fa0.870.960.600.990.821.081.02 1.120.780.701.010.150.721.010.780.811.031.030.840.721.100.15
fr1.952.180.912.111.402.452.341.29 1.701.202.300.251.662.091.761.322.342.321.681.561.720.33
he1.872.090.922.041.382.342.231.282.40 1.192.180.241.582.031.691.292.232.231.651.511.680.31
hu1.251.380.761.451.121.561.491.071.601.12 1.460.191.061.431.131.061.481.501.191.051.380.21
it1.972.210.922.141.412.492.371.302.571.731.22 0.251.682.121.791.332.382.361.711.581.740.33
ja1.301.440.761.481.111.611.541.111.651.160.941.50 1.091.461.151.011.531.541.211.091.410.22
ko1.942.160.892.091.392.432.321.262.511.701.202.280.24 2.071.761.312.332.311.671.541.700.33
nl1.761.980.901.961.372.212.131.282.311.591.182.100.241.51 1.611.292.132.131.601.451.700.30
pl1.882.100.892.041.372.362.251.242.431.651.162.210.241.602.01 1.302.252.251.621.511.690.32
pt1.091.220.701.241.001.381.301.011.410.990.851.280.160.931.251.01 1.301.311.040.921.280.19
pt-br1.962.200.922.121.412.462.361.302.551.721.212.320.251.672.091.771.32 2.341.701.571.740.33
ro1.952.190.932.121.422.472.341.302.531.721.232.300.251.662.101.771.332.34 1.701.571.740.33
ru1.651.840.861.831.312.061.971.222.121.481.131.930.231.391.831.481.221.961.97 1.351.630.28
tr1.751.960.871.951.332.212.101.222.281.571.152.070.241.481.921.591.252.102.101.56 1.650.30
vi1.081.200.711.261.001.361.291.021.390.970.841.270.170.911.250.990.961.291.301.050.91 0.19
zh-cn1.932.150.912.081.392.342.311.282.481.691.192.260.241.632.051.741.302.302.291.661.531.70