臺灣博碩士論文加值系統

用戶如何對照片使用濾鏡可能會透露出用戶的偏好或心理狀態，因此可能需要使用照片濾鏡分類方法來實現未來的大規模分析。我們收集專門用於照片濾鏡分類的數據集，並且採用遷移學習(transfer learning)技術將預先以物件分類為目標的深度模型轉換為適合於濾鏡分類的模型。基於準確的分類結果，我們構建了一個伴隨矩陣(co-occurrence matrix)，以描述視覺特徵與濾鏡種類之間的頻率關係，最後經由非負矩陣分解(non-negative matrix factorization)實現濾鏡推薦。此方法不需太多的手動標記，當有更多的訓練數據時，它可以很容易地擴展。

在實驗評測中，我們驗證了各種常見的卷積神經網路的效能，並進行了一系列實驗研究，以證明濾鏡分類與遷移學習的有效性。我們還證明所提出的濾鏡推薦系統具有不錯的性能。

Because how users employ filters to photos may reveal user’s preference or mental state, a photo filter classification method is potentially demanded to enable future large-scale analysis.
We collect a dataset specifically for photo filter classification, and adopt the transfer learning technique to transform deep models pre-trained for object classification into that suitable for photo filter classification. Based on accurate classification results, we build a co-occurrence matrix to describe the frequency between visual information and filter types. Without much manual labeling, a filter recommendation approach is implemented via non-negative matrix decomposition. It can be easily extended when more training data are available.

In the evaluation, we verify different common convolutional neural networks and show performance comparison.
A series of experimental studies are conducted to demonstrate effectiveness of filter classification with transfer learning. We also demonstrate the proposed filter recommendation achieves encouraging performance.

1 Introduction................................. 1
1.1 Motivation................................. 1
1.2 SystemOverview............................. 2
1.3 Contributions............................... 3
1.4 ThesisOrganization............................ 3
2 RelatedWorks................................. 4
2.1 PhotoFilter................................ 4
2.2 RecommendationSystem......................... 6
2.3 ShortSummary.............................. 7
3 PhotoFilterClassification................................. 8
3.1 Datasets.................................. 8
3.2 TransferLearningfromPre-trainedModels............... 9
4 PhotoFilterRecommendation................................. 11
4.1 Dataset.................................. 12
4.2 MatrixFactorization........................... 12
4.3 VisualInformation............................ 15
4.3.1 AutoFeatures........................... 15
4.3.2 PlaceInformation......................... 16
4.3.3 AestheticsInformation...................... 16
4.3.4 ObjectInformation........................ 17
4.3.5 Fusion............................... 17
5 Experments................................. 19
5.1 FilterClassification............................ 19
5.2 FilterRecommendation.......................... 23
6 ConclusionandFutureWorks................................. 28
6.1 Conclusion................................ 28
6.2 FutureWorks............................... 28
References................................. 30
Appendix................................. 34

[1] Wei-TseSun,Ting-HsuanChao,Yin-HsiKuo,andWinstonH.Hsu.Photofil-
ter recommendationbycategory-awareaestheticlearning. IEEE Transactionson
Multimedia, 19(8):1870–1880,2017.
[2] Saeideh Bakhshi,DavidA.Shamma,LyndonKennedy,andEricGilbert.Whywe
filter ourphotosandhowitimpactsengagement.In ProceedingsofInternational
AAAI ConferenceonWebandSocialMedia, 2015.
[3] AndrewG.ReeceandChristopherM.Danforth.Instagramphotosrevealpredic-
tivemarkersofdepression. EPJ DataScience, 6(1):15,2017.
[4] Naila Murray,LucaMarchesotti,andFlorentPerronnin.Ava:Alarge-scale
database foraestheticvisualanalysis.In ProceedingsofIEEEConferenceon
Computer VisionandPatternRecognition, pages2408–2415.IEEE,2012.
[5] Gediminas AdomaviciusandAlexanderTuzhilin.Towardthenextgenerationof
recommender systems:Asurveyofthestate-of-the-artandpossibleextensions.
IEEE TransactionsonKnowledgeandDataEngineering, 17(6):734–749,2005.
[6] GregLinden,BrentSmith,andJeremyYork.Amazon.comrecommendations:
Item-to-item collaborativefiltering. IEEE InternetComputing, 7(1):76–80,2003.
[7] Wei-TaChuandWei-HanHuang.Culturaldifferenceandvisualinformationon
hotel ratingprediction. WorldWideWeb, 20(4):595–619,2017.
[8] Zhibo Wang,JilongLiao,QingCao,HairongQi,andZhiWang.Friendbook:a
semantic-based friendrecommendationsystemforsocialnetworks. IEEE Trans-
actions onMobileComputing, 14(3):538–551,2015.
[9] Wei-TaChuandYa-LunTsai.Ahybridrecommendationsystemconsideringvi-
sual informationforpredictingfavoriterestaurants. WorldWideWeb, 20(6):1313–
1331, 2017.
[10] Chung-HuaChuandSe-HsienWu.Achineserestaurantrecommendationsystem
based onmobilecontext-awareservices.In ProceedingsofIEEEInternational
ConferenceonMobileDataManagement, volume2,pages116–118,2013.
[11] Felicia Cordeiro,ElizabethBales,ErinCherry,andJamesFogarty.Rethinkingthe
mobile foodjournal:Exploringopportunitiesforlightweightphoto-basedcapture.
In ProceedingsofACMConferenceonHumanFactorsinComputingSystems,
pages 3207–3216.ACM,2015.
[12] Michael J.PazzaniandDanielBillsus.Content-basedrecommendationsystems.
The AdaptiveWeb, pages325–341,2007.
[13] PasqualeLops,MarcoDeGemmis,andGiovanniSemeraro.Content-basedrec-
ommender systems:Stateoftheartandtrends. Recommender SystemsHandbook,
pages 73–105,2011.
[14] John S.Breese,DavidHeckerman,andCarlKadie.Empiricalanalysisofpre-
dictivealgorithmsforcollaborativefiltering.In ProceedingsofConferenceon
Uncertainty inArtificialIntelligence, pages43–52,1998.
[15] Jia Deng,WeiDong,RichardSocher,Li-JiaLi,KaiLi,andLiFei-Fei.Imagenet:
A large-scalehierarchicalimagedatabase.In ProceedingsofIEEEConferenceon
Computer VisionandPatternRecognition, pages248–255,2009.
[16] Tsung-YiLin,MichaelMaire,SergeBelongie,JamesHays,PietroPerona,Deva
Ramanan, PiotrDoll´ar,andCLawrenceZitnick.Microsoftcoco:Commonob-
jects incontext.In ProceedingsofEuropeanConferenceonComputerVision,
pages 740–755.Springer,2014.
[17] Lorien YPratt.Discriminability-basedtransferbetweenneuralnetworks.In Pro-
ceedings ofAdvancesinNeuralInformationProcessingSystems, pages204–211,
1993.
[18] AlexKrizhevsky,IlyaSutskever,andGeoffreyEHinton.Imagenetclassification
with deepconvolutionalneuralnetworks.In ProceedingsofAdvancesinNeural
Information ProcessingSystems, pages1097–1105,2012.
[19] K.SimonyanandA.Zisserman.Verydeepconvolutionalnetworksforlarge-
scale imagerecognition.In ProceedingsofInternationalConferenceonLearning
Representations, 2015.
[20] KaimingHe,XiangyuZhang,ShaoqingRen,andJianSun.Deepresiduallearning
for imagerecognition.In ProceedingsofIEEEConferenceonComputerVision
and PatternRecognition, pages770–778,2016.
[21] YehudaKoren,RobertBell,andChrisVolinsky.Matrixfactorizationtechniques
for recommendersystems. Computer, 42(8):30–37,2009.
[22] MichaelWBerry,MurrayBrowne,AmyNLangville,VPaulPauca,andRobertJ
Plemmons. Algorithmsandapplicationsforapproximatenonnegativematrixfac-
torization. Computational StatisticsandDataAnalysis, 52(1):155–173,2007.
[23] Amy N.Langville,CarlD.Meyer,RussellAlbright,JamesCox,andDavidDul-
ing. Algorithms,initializations,andconvergenceforthenonnegativematrixfac-
torization. arXiv preprintarXiv:1407.7299, 2014.
[24] Daniel D.LeeandH.SebastianSeung.Algorithmsfornon-negativematrixfac-
torization. In ProceedingsofAdvancesinNeuralInformationProcessingSystems,
pages 556–562,2001.
[25] Zhonglong Zheng,JieYang,andYitanZhu.Initializationenhancerfornon-
negativematrixfactorization. Engineering ApplicationsofArtificialIntelligence,
20(1):101–110, 2007.
[26] Bolei Zhou,AgataLapedriza,AdityaKhosla,AudeOliva,andAntonioTorralba.
Places: A10millionimagedatabaseforscenerecognition. IEEE Transactionson
PatternAnalysisandMachineIntelligence, 40(6):1452–1464,2018.
[27] Xin Lu,ZheLin,HailinJin,JianchaoYang,andJamesZ.Wang.Rapid:Rating
pictorial aestheticsusingdeeplearning.In ProceedingsoftheACMInternational
ConferenceonMultimedia, pages457–466,2014.
[28] HosseinTalebiandPeymanMilanfar.Nima:Neuralimageassessment. IEEE
TransactionsonImageProcessing, 27(8):3998–4011,2018.
[29] JosephRedmonandAliFarhadi.Yolov3:Anincrementalimprovement. arXiv
preprintarXiv:1804.02767, 2018.
[30] Yang-YuFan,ShuLiu,BoLi,ZheGuo,AshokSamal,JunWan,andStanZ
Li. Labeldistribution-basedfacialattractivenesscomputationbydeepresidual
learning. IEEE TransactionsonMultimedia, 20(8):2196–2208,2017.