Seminar Topics and Projects

(1)

Seminar Topics and Projects

Giuseppe Attardi Giuseppe Attardi

Dipartimento di Informatica Dipartimento di Informatica

Università di Pisa Università di Pisa

Università di Pisa

(2)

Deep Learning Tokenizer Deep Learning Tokenizer

 Depling 2016 challenge requires tokenizer for Depling 2016 challenge requires tokenizer for any of the Universal Dependency TreeBank any of the Universal Dependency TreeBank

 Build a DL tokenizer using Keras based on Build a DL tokenizer using Keras based on the approach of:

the approach of:

 Basile, Valerio and Bos, Johan and Evang, Kilian A General-Purpose Machine Learning Method for Tokenization and Sentence Boundary Detection (2013), http://gmb.let.rug.nl/elephant/

(3)

Deep Learning POS for UD Deep Learning POS for UD

 Build a DL POS using CNN, for example a Build a DL POS using CNN, for example a LSTM that uses word embeddings and

LSTM that uses word embeddings and possible charcater embeddings.

possible charcater embeddings.

(4)

Deep Learning Morph Deep Learning Morph

Analyzer Analyzer

 Build a DL morphological analyzer that Build a DL morphological analyzer that copmutes the morphology of each word, copmutes the morphology of each word, using Keras and charcaher embeddings.

using Keras and charcaher embeddings.

(5)

UD extensions UD extensions

 Write scripts to extract additional relations Write scripts to extract additional relations from the analysis of UD parse trees

from the analysis of UD parse trees

(6)

Convolutional Networks for Sentiment Convolutional Networks for Sentiment

Analysis Analysis

 Annotated Data: SemEval training setAnnotated Data: SemEval training set

 Unannotated Data: 50 million tweetsUnannotated Data: 50 million tweets

 Code: DeepNL, https://github.com/attardi/Code: DeepNL, https://github.com/attardi/

deepnl deepnl

 Article: A. Severyn, A. Moschitti.UNITNArticle: A. Severyn, A. Moschitti.UNITN

: Training Deep Convolutional Neural Network : Training Deep Convolutional Neural Network

for Twitter Sentiment for Twitter Sentiment

Classification Classification

(7)

POS tagging using Word POS tagging using Word

Embeddings Embeddings

 Data: Evalita 2016Data: Evalita 2016

 Embeddings: Embeddings:

http://tanl.di.unipi.it/embeddings/

 Article: Stratos, M. Collins. Simple Semi-Article: Stratos, M. Collins. Simple Semi- Supervised POS Tagging.

Supervised POS Tagging.

http://www.cs.columbia.edu/~stratos/research http://www.cs.columbia.edu/~stratos/research

/naacl15semipos.pdf /naacl15semipos.pdf

(8)

Negation/Speculation Negation/Speculation

Extraction Extraction

 Determine the scope of negative or Determine the scope of negative or speculative statements:

speculative statements:

 The lyso-platelet had no effect

 MnlI-AluI could suppress the basal-level activity

 Approach:Approach:

 Classifier for identifying cues

 Classifier to determine scope

 DataData

 BioScope collection

(9)

Corpus of Product Reviews Corpus of Product Reviews

 Download reviews from online shopsDownload reviews from online shops

 Classify as positive/negative according to Classify as positive/negative according to stars

stars

 Train classifier to assign scoreTrain classifier to assign score

(10)

Relation Extraction Relation Extraction

 Exploit word embeddings as features + extra Exploit word embeddings as features + extra hand-coded features

hand-coded features

 Use the Factor Based Compositional Use the Factor Based Compositional Embedding Model (FCM)

Embedding Model (FCM)

http://www.cs.jhu.edu/~mrg/publications/finer http://www.cs.jhu.edu/~mrg/publications/finer

e-naacl-2015.pdf e-naacl-2015.pdf

 SemEval 2014 Relation Extraction dataSemEval 2014 Relation Extraction data

(11)

Entity Linking with Embeddings Entity Linking with Embeddings

 Experiment with technique:Experiment with technique:

R. Blanco, G. Ottaviano, E. Meiji. 2014. Fast and Space-Efficient Entity Linking in Queries.

labs.yahoo.com/_c/uploads/WSDM-2015-blanco.pdf

(12)

Extraction of Semantic Extraction of Semantic

Hierarchies Hierarchies

 Use word embeddings as measure of semantic distance

 Use Wikipedia as source of text

 http://ir.hit.edu.cn/~jguo/papers/acl2014- hypernym.pdf

Aconitum Ranuncolacee

Plant Organism

(13)

Seminars

(14)

Neural Reasoning Neural Reasoning

 B. Peng, Z. Lu, H. Li, K.F. WongToward B. Peng, Z. Lu, H. Li, K.F. WongToward Neural Network-based Reasoning

Neural Network-based Reasoning

 A. Kumar et al.A. Kumar et al.Ask Me Anything: Dynamic Ask Me Anything: Dynamic Memory Networks for Natural Language Memory Networks for Natural Language

Processing Processing

(15)

Question Answering Question Answering

 Bowl Competition (QANTA vs Jennings)Bowl Competition (QANTA vs Jennings)

 https://www.youtube.com/watch?

v=kTXJCEvCDYk

 Iyyer et al. 2014: A Neural Network for

Factoid Question Answering over Paragraphs

 IBM Watson:IBM Watson:

 http://www.aaai.org/Magazine/Watson/watson.php

 TAC:TAC:

 http://www.nist.gov/tac/2008/qa/index.html

(16)

Image Understanding Image Understanding

 H. Y. Gao et al. Are You Talking to a H. Y. Gao et al. Are You Talking to a Machine? Dataset and Methods for Machine? Dataset and Methods for

Multilingual Image Question Answering, Multilingual Image Question Answering,

NIPS, 2015.

(17)

Deep Learning Deep Learning

Applications Applications

 Character RNNs on text and code

 http://karpathy.github.io/2015/05/21/rnn-effec8veness/

 Morphology

 Better Word Representations with Recursive Neural Networks for Morphology – Luong et al.

 Polysemous words

 Improving Word Representa8ons Via Global Context And Multiple Word Prototypes by Huang et al. 2012

 Natural language Inference (Logic)

 Question Answering

 Image – Sentence mapping

(18)

Entity Linking Entity Linking

 Entity Kierarchy EmbeddingsEntity Kierarchy Embeddings

 http://www.cs.cmu.edu/~zhitingh/data/acl15entity.

pdf

(19)

Deep Learning tsunami over Deep Learning tsunami over NLP NLP

 C. Manning. 2015. C. Manning. 2015.

http://www.mitpressjournals.org/doi/pdf/10.11 http://www.mitpressjournals.org/doi/pdf/10.11

62/COLI_a_00239 62/COLI_a_00239

(20)

Opinion Mining Opinion Mining

 B. Liu. Sentiment Analisis and Subjectivity. B. Liu. Sentiment Analisis and Subjectivity.

2010. Handbook of NLP.

http://www.cs.uic.edu/~liub/FBS/NLP- http://www.cs.uic.edu/~liub/FBS/NLP-

handbook-sentiment-analysis.pdf handbook-sentiment-analysis.pdf

(21)

Semantic Role Labeling Semantic Role Labeling

 http://ufal.mff.cuni.cz/conll2009-st/task-http://ufal.mff.cuni.cz/conll2009-st/task- description.html

description.html

(22)

DL for NLP DL for NLP

 Neural Machine TranslationNeural Machine Translation

 D. Bahdanau, K. Cho, Y. Bengio. Neural machine translation by jointly learning to align and translate.

http://arxiv.org/pdf/1409.0473v6

 Natural Language from scratchNatural Language from scratch

 Zhang, X., & LeCun, Y. (2015). Text Understanding from Scratch.

http://arxiv.org/abs/1502.01710

Seminar Topics and Projects