German FinBERT For Sentiment Analysis (Pre trained From Scratch Version, Fine Tuned for Financial Sentiment Analysis) German FinBERT is a BERT language model focusing on the financial domain within the German language. In my paper, I describe in more detail the steps taken to train the model and show that it outperforms its generic benchmarks for finance specific downstream tasks. This model is the pre trained from scratch version of German FinBERT, after fine tuning on a translated version of the financial news phrase bank of Malo et al. (2013). The data is available here. Overview Author Moritz Scherrmann Paper: here Architecture: BERT base Language: German Specialization: Financial sentiment Base model: German FinBert SC Fine tuning I fine tune the model using the 1cycle policy of Smith and Topin (2019). I use the Adam optimization method of Kingma and Ba (2014) with standard parameters.I run a grid search on the evaluation set to find the best hyper parameter setup. I test different values for learning rate, batch size and number of epochs, following the suggestions of Chalkidis et al. (2020). I repeat the fine tuning for each setup five times with different seeds, to avoid get…
We use cookies for essential functionality and analytics. You can accept or reject analytics cookies.Cookie policy