10
[email protected] [email protected] [email protected] [email protected] Convolutional Neural Networks for Sentiment Analysis in Persian Social Media Morteza Rohanian, MSc 1 , Mostafa Salehi, PhD 2 , Ali Darzi, PhD 3 , Vahid Ranjbar, PhD 1- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected] 2- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected] 3- Faculty of Literature and Humanities, University of Tehran, Tehran, Iran, Email: [email protected] 1- Department of Computer Enigneering, Yazd University, Yazd, Iran, Email: [email protected] Abstract: With the social media engagement on the rise, the resulting data can be used as a rich resource for analyzing and understanding different phenomena around us. A sentiment analysis system employs these data to find the attitude of social media users towards certain entities in a given document. In this paper we propose a sentiment analysis method for Persian text using Convolutional Neural Network (CNN), a feedforward Artificial Neural Network, that categorize sentences into two and five classes (considering their intensity) by applying a layer of convolution over input data through different filters. We evaluated the method on three different datasets of Persian social media texts using Area under Curve metric. The final results show the advantage of using CNN over earlier attempts at developing traditional machine learning methods for Persian texts sentiment classification especially for short texts. Keywords: Sentiment Analysis, Social Media, Convolutional Neural Network, Sentiment Intensity , Short Texts نام نو ي سنده مسئول: نشان ي نو ي سنده مسئول:

Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

  • Upload
    others

  • View
    11

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

[email protected]

[email protected]

[email protected]

[email protected]

Convolutional Neural Networks for Sentiment Analysis in Persian

Social Media

Morteza Rohanian, MSc1, Mostafa Salehi, PhD2, Ali Darzi, PhD3, Vahid Ranjbar, PhD

1- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected]

2- Faculty of New Sciences and Technologies, University of Tehran, Tehran, Iran, Email: [email protected]

3- Faculty of Literature and Humanities, University of Tehran, Tehran, Iran, Email: [email protected]

1- Department of Computer Enigneering, Yazd University, Yazd, Iran, Email: [email protected]

Abstract: With the social media engagement on the rise, the resulting data can be used as a rich resource for analyzing and understanding

different phenomena around us. A sentiment analysis system employs these data to find the attitude of social media users towards certain

entities in a given document. In this paper we propose a sentiment analysis method for Persian text using Convolutional Neural Network

(CNN), a feedforward Artificial Neural Network, that categorize sentences into two and five classes (considering their intensity) by

applying a layer of convolution over input data through different filters. We evaluated the method on three different datasets of Persian

social media texts using Area under Curve metric. The final results show the advantage of using CNN over earlier attempts at developing

traditional machine learning methods for Persian texts sentiment classification especially for short texts.

Keywords: Sentiment Analysis, Social Media, Convolutional Neural Network, Sentiment Intensity , Short Texts

سنده مسئول:ينام نو

سنده مسئول: ينو ينشان

Page 2: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

-

:

3

4

5)CNN(6

7

-

-

-

8

Page 3: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

9

1011-

12SVM

SVMيبنددسته

استخراج يهايژگيمختلف با استفاده از و يهاا در کالسيکردن اش

شده است

-

13RNN

RNN

RNN

14

15

[28 ]

SVM

-

LSTM

16

17

-

(CNN)

18

19

20

Page 4: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

21

𝑥 ∈ 𝑅𝑠

𝑠𝑑 ∈

𝑅𝑛×𝑠𝑛

𝑊 ∈ 𝑅ℎ×𝑠ℎ

*

𝑊 ∗ 𝑑𝑗:𝑗+ℎ−1 = ∑ ∑ 𝑊𝑖,𝑘𝑑𝑖,𝑘𝑠−1𝑘=0

𝑗+ℎ−1𝑖=𝑗 .

ℎ𝑑𝑗:𝑗+ℎ−1

𝑓𝑐𝑗

𝑐𝑗 = 𝑓(𝑑𝑗:𝑗+ℎ−1 + 𝑏),

𝑏22

𝑊

𝑐(𝑊) = [𝑐1, 𝑐2, … , 𝑐𝑛−ℎ+1].

23

24

25

Page 5: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

�̂�𝑊 = max 𝑐(𝑊)𝑖.

26

�̂�𝑊 = [�̂�𝑊1 , … , �̂�𝑊𝑘]𝑇,

𝑇 = {1, … , 𝑘}

يه الحاق، ما عمل الحاق حداکثريدر ال

ه کانولوشن و به يمختلف حاصل از ال يهايژگيب ويترک يرا برا

دن به يرس يم. برايدهيک بردار با بعد ثابت انجام ميآوردن وجود

يارهيوند زنجيه کانولوشن را پيال يهايصاف يبردار جمله خروج

يهاتعداد نگاشت ،شتر باشديها بيم. هرچه تعداد صافيدهيم

ن مقدار در يشتريب ،يژگيشود و از هر نگاشت ويشتر ميب يژگيو

يسراسر يژگيبردار و شود و بهيانتخاب م يمرحله الحاق حداکثر

توان گفت کهيپس م شود.ياضافه م

،

𝑈 ∈ 𝑅2×𝑘𝑏𝑈 ∈ 𝑅2

𝑦𝑗 = 𝑈𝑗 �̂�𝑊 + 𝑏𝑗𝑈

𝑐𝑤𝑈𝑗𝑗 𝑈𝑏𝑗𝑈𝑗𝑏𝑈𝑌𝑗

𝑗𝑑

𝑃(𝑌𝑗 = 1|𝑑, 𝑊, 𝑏𝑈) =𝑒

𝑦𝑗

∑ 𝑒𝑦𝑖𝑖,

𝐷

− ∑ log( 𝑃(𝑌𝑃𝑂𝑆𝑑 − 1|𝑑, 𝑊, 𝑏𝑈))𝑑∈𝐷

− ∑ log( 𝑃(𝑌𝑃𝑂𝑆𝑑 − 2|𝑑, 𝑊, 𝑏𝑈))𝑑∈𝐷

𝑑, 𝑊, 𝑏𝑈

CNN

27

𝑠

-

-

مجموعه داده -4-1

@username

http://t.co/urlURLs

2007

#tweet

/\

Page 6: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

يسازادهيپ -4-2

GPU

KerasTheano][

scikit-learn]25[

معيار ارزيابي -4-3

Y

X

31AUC

Page 7: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

CNN

RNN

يآنتروپ

نهيشيب

SVM

-

Page 8: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

-

-

-

-

[1] Greenwood, S., A. Perrin, and M. Duggan. "Social

media update 2016: Facebook usage and engagement

is on the rise, while adoption of other platforms holds

steady." Pew Research Center (2016).

[2] Mander, Jason. "Daily time spent on social networks

rises to 1.72 hours." London: Global Web Index (2015).

[3] Liu, Bing. "Sentiment analysis and opinion mining."

Synthesis lectures on human language technologies 5,

no. 1 (2012): 1-167.

[4] Pang, Bo, Lillian Lee, and Shivakumar Vaithyanathan.

"Thumbs up?: sentiment classification using machine

Page 9: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

learning techniques." In Proceedings of the ACL-02

conference on Empirical methods in natural language

processing-Volume 10, pp. 79-86. Association for

Computational Linguistics, 2002.

[5] Tripathy, Abinash, Ankit Agrawal, and Santanu Kumar

Rath. "Classification of sentiment reviews using n-

gram machine learning approach." Expert Systems with

Applications 57 (2016): 117-126.

[6] Mullen, Tony, and Nigel Collier. "Sentiment Analysis

using Support Vector Machines with Diverse

Information Sources." In EMNLP, vol. 4, pp. 412-418.

2004.

[7] Agarwal, Basant, and Namita Mittal. "Machine

learning approach for sentiment analysis." In

Prominent Feature Extraction for Sentiment Analysis,

pp. 21-45. Springer International Publishing, 2016.

[8] Poria, Soujanya, Haiyun Peng, Amir Hussain, Newton

Howard, and Erik Cambria. "Ensemble application of

convolutional neural networks and multiple kernel

learning for multimodal sentiment analysis."

Neurocomputing (2017).

[9] Dong, Li, Furu Wei, Chuanqi Tan, Duyu Tang, Ming

Zhou, and Ke Xu. "Adaptive Recursive Neural

Network for Target-dependent Twitter Sentiment

Classification." In ACL (2), pp. 49-54. 2014.

[10] Kim, Yoon. "Convolutional neural networks for

sentence classification." arXiv preprint

arXiv:1408.5882 (2014).

[11] Zhang, Ye, and Byron Wallace. "A sensitivity analysis

of (and practitioners' guide to) convolutional neural

networks for sentence classification." arXiv preprint

arXiv:1510.03820 (2015).

[12] Jaynes, Edwin T. "Information theory and statistical

mechanics." Physical review 106, no. 4 (1957): 620.

[13] Neethu, M. S., and R. Rajasree. "Sentiment analysis in

twitter using machine learning techniques." In

Computing, Communications and Networking

Technologies (ICCCNT), 2013 Fourth International

Conference on, pp. 1-5. IEEE, 2013.

[14] Cortes, Corinna, and Vladimir Vapnik. "Support-

vector networks." Machine learning 20, no. 3 (1995):

273-297.

[15] Dos Santos, Cícero Nogueira, and Maira Gatti. "Deep

Convolutional Neural Networks for Sentiment

Analysis of Short Texts." In COLING, pp. 69-78. 2014.

[16] Zhang, Yu, Mengdong Chen, Lianzhong Liu, and

Yadong Wang. "An effective convolutional neural

network model for Chinese sentiment analysis." In AIP

Conference Proceedings, vol. 1836, no. 1, p. 020085.

AIP Publishing, 2017.

[17] Cieliebak, Mark, Jan Deriu, Dominic Egger, and Fatih

Uzdilli. "A Twitter Corpus and Benchmark Resources

for German Sentiment Analysis." SocialNLP 2017

(2017): 45.

[18] Socher, Richard, Alex Perelygin, Jean Y. Wu, Jason

Chuang, Christopher D. Manning, Andrew Y. Ng, and

Christopher Potts. "Recursive deep models for

semantic compositionality over a sentiment treebank."

In Proceedings of the conference on empirical methods

in natural language processing (EMNLP), vol. 1631,

p. 1642. 2013.

[19] Collobert, Ronan, Jason Weston, Léon Bottou,

Michael Karlen, Koray Kavukcuoglu, and Pavel

Kuksa. "Natural language processing (almost) from

scratch." Journal of Machine Learning Research 12,

no. Aug (2011): 2493-2537.

[20] Wang, Keze, Xiaolong Wang, Liang Lin, Meng Wang,

and Wangmeng Zuo. "3D human activity recognition

with reconfigurable convolutional neural networks." In

Proceedings of the 22nd ACM international

conference on Multimedia, pp. 97-106. ACM, 2014.

[21] Mikolov, Tomas, Ilya Sutskever, Kai Chen, Greg S.

Corrado, and Jeff Dean. "Distributed representations of

words and phrases and their compositionality." In

Advances in neural information processing systems,

pp. 3111-3119. 2013.

[22] Bengio, Yoshua, Réjean Ducharme, Pascal Vincent,

and Christian Jauvin. "A neural probabilistic language

model." Journal of machine learning research 3, no.

Feb (2003): 1137-1155.

[23] Bojanowski, Piotr, Edouard Grave, Armand Joulin,

and Tomas Mikolov. "Enriching word vectors with

subword information." arXiv preprint

arXiv:1607.04606 (2016).

[24] Bagheri, Ayoub, and Mohamad Saraee. "Persian

Sentiment Analyzer: A Framework based on a Novel

Feature Selection Method." International Journal of

Artificial Intelligence™ 12, no. 2 (2014): 115-129.

[25] Pedregosa, Fabian, Gaël Varoquaux, Alexandre

Gramfort, Vincent Michel, Bertrand Thirion, Olivier

Grisel, Mathieu Blondel et al. "Scikit-learn: Machine

learning in Python." Journal of Machine Learning

Research 12, no. Oct (2011): 2825-2830.

[26] Bergstra, James, Olivier Breuleux, Frédéric Bastien,

Pascal Lamblin, Razvan Pascanu, Guillaume

Desjardins, Joseph Turian, David Warde-Farley, and

Yoshua Bengio. "Theano: A CPU and GPU math

compiler in Python." In Proc. 9th Python in Science

Conf, pp. 1-7. 2010.

[27] Hajmohammadi, Mohammad Sadegh, and Roliana

Ibrahim. "A svm-based method for sentiment analysis

in persian language." In International Conference on

Graphic and Image Processing (ICGIP 2012), vol.

8768, p. 876838. International Society for Optics and

Photonics, 2013.

[28] Basiri, Mohammad Ehsan, Ahmad Reza Naghsh-

Nilchi, and Nasser Ghassem-Aghaee. "A framework

for sentiment analysis in persian." Open Transactions

on Information Processing 1, no. 3, pp 1-14 ,2014.

[29] Roshanfekr, Behnam, Shahram Khadivi, and

Mohammad Rahmati. "Sentiment analysis using deep

learning on Persian texts." In 2017 Iranian Conference

on Electrical Engineering (ICEE), pp. 1503-1508.

IEEE, 2017.

Page 10: Convolutional Neural Networks for Sentiment Analysis in ... · Zhou, and Ke Xu. "Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification." In ACL (2),

[]

.

[]

[]

1 Subjective Opinions

2 Objective Opinions 3 Supervised Learning 4 Generic

5 Convolutional Neural Networks 6 Feedforward 7 Word Embedding 8 Naïve Bayes classifier 9 Maximum Entropy Classifier (MEC) 1 0 Exponential Model 1 1 Principle of Maximum Entropy 1 2 Support Vector Machines (SVMs) 1 3 Recurrent Neural Networks 1 4 Semantic Role Labeling 1 5 Chunking 1 6 Representation Learning

[]

RTDGPS

RNNPSO

اهسيرنويز

1 7 Unsupervised Learning 1 8 Grid 1 9 Filters 2 0 Feature Maps 2 1 Max-pooling 2 2 Bias 2 3 Channel 2 4 Sentence Vector 2 5 Softmax 2 6 Global Feature Vector

2 7 Distributed Representation of Words 2 8 Validation Loss 2 9 Dropout 3 0 Adadelta 3 1 Area Under Curve