FAOLLASHTIRISH FUNKSIYALARI: SIGMOID, TANH, RELU VA BOSHQALAR
Keywords:
Kalit so'zlar: faollashtirish funksiyalari, neyron tarmoqlar, chuqur o'rganish, sigmoid funksiyasi, giperbolik tangens, to'g'rilovchi chiziqli birlik, gradient tushish, xatoni orqaga tarqatish, nochiziqli o'zgarishlar, mashinali o'rganishAbstract
ANNOTATSIYA: Ushbu maqola sun'iy neyron tarmoq arxitekturalarida
faollashtirish funksiyalarining har tomonlama tadqiqotiga va ularning chuqur o'rganish
jarayonlaridagi muhim roliga bag'ishlangan. Tadqiqot klassik faollashtirish
funksiyalarining matematik xususiyatlarini batafsil tahlil qilishni, jumladan sigmoid
funksiyasi, giperbolik tangens, to'g'rilovchi chiziqli birlik va ushbu funksiyalarning
zamonaviy modifikatsiyalarini o'z ichiga oladi. Tadqiqot neyron tarmoqlardagi
nochiziqli o'zgarishlarning nazariy asoslarini, gradient tarqalish mexanizmlarini,
yo'qolib borayotgan va portlovchi gradient muammolarini, shuningdek turli
faollashtirish funksiyalarining hisoblash samaradorligini qamrab oladi.
References
FOYDALANILGAN ADABIYOTLAR
1. Goodfellow I., Bengio Y., Courville A. Deep Learning. MIT Press, 2016.
2. Glorot X., Bengio Y. “Understanding the difficulty of training deep feedforward
neural networks.” AISTATS, 2010.
3. Ramachandran P., Zoph B., Le Q.V. “Swish: a Self-Gated Activation Function.”
Google Brain, 2017.
4. He K., Zhang X., Ren S., Sun J. “Delving Deep into Rectifiers: Surpassing
Human-Level Performance on ImageNet Classification.” ICCV, 2015.
5. Clevert D., Unterthiner T., Hochreiter S. “Fast and Accurate Deep Network
Learning by Exponential Linear Units (ELUs).” ICLR, 2016.
6. Hendrycks D., Gimpel K. “Gaussian Error Linear Units (GELUs).”
arXiv:1606.08415, 2016.
7. Maas A.L., Hannun A.Y., Ng A.Y. “Rectifier Nonlinearities Improve Neural
Network Acoustic Models.” ICML, 2013.
8. Haykin S. Neural Networks and Learning Machines. Pearson, 2009.
