April 11, 2023

81% Match

Evelyn Herberg

These lecture notes provide an overview of Neural Network architectures from a mathematical point of view. Especially, Machine Learning with Neural Networks is seen as an optimization problem. Covered are an introduction to Neural Networks and the following architectures: Feedforward Neural Network, Convolutional Neural Network, ResNet, and Recurrent Neural Network.

Find SimilarView on arXiv

May 3, 2024

80% Match

Corrado Coppola, Lorenzo Papa, Marco Boresta, ... , Palagi Laura

The paper aims to investigate relevant computational issues of deep neural network architectures with an eye to the interaction between the optimization algorithm and the classification performance. In particular, we aim to analyze the behaviour of state-of-the-art optimization algorithms in relationship to their hyperparameters setting in order to detect robustness with respect to the choice of a certain starting point in ending on different local solutions. We conduct exten...

Find SimilarView on arXiv

April 6, 2023

80% Match

Garrett Bingham

Automated machine learning (AutoML) methods improve upon existing models by optimizing various aspects of their design. While present methods focus on hyperparameters and neural network topologies, other aspects of neural network design can be optimized as well. To further the state of the art in AutoML, this dissertation introduces techniques for discovering more powerful activation functions and establishing more robust weight initialization for neural networks. These contr...

Find SimilarView on arXiv

December 16, 2023

80% Match

Luis Balderas, Miguel Lastra, José M. Benítez

Deep learning models have been widely used during the last decade due to their outstanding learning and abstraction capacities. However, one of the main challenges any scientist has to face using deep learning models is to establish the network's architecture. Due to this difficulty, data scientists usually build over complex models and, as a result, most of them result computationally intensive and impose a large memory footprint, generating huge costs, contributing to clima...

Find SimilarView on arXiv

June 18, 2024

79% Match

Mark Potanin, Kirill Vayser, Vadim Strijov

Neural network structures have a critical impact on the accuracy and stability of forecasting. Neural architecture search procedures help design an optimal neural network according to some loss function, which represents a set of quality criteria. This paper investigates the problem of neural network structure optimization. It proposes a way to construct a loss function, which contains a set of additive elements. Each element is called the regularizer. It corresponds to some ...

Find SimilarView on arXiv

July 8, 2022

79% Match

Olga Lukyanova, Oleg Nikitin, Alex Kunin

In this article, we propose the approach to structural optimization of neural networks, based on the braid theory. The paper describes the basics of braid theory as applied to the description of graph structures of neural networks. It is shown how networks of various topologies can be built using braid structures between layers of neural networks. The operation of a neural network based on the braid theory is compared with a homogeneous deep neural network and a network with ...

Find SimilarView on arXiv

June 2, 2022

79% Match

Isak Potgieter, Christopher W. Cleghorn, Anna S. Bosman

This study investigates the use of local optima network (LON) analysis, a derivative of the fitness landscape of candidate solutions, to characterise and visualise the neural architecture space. The search space of feedforward neural network architectures with up to three layers, each with up to 10 neurons, is fully enumerated by evaluating trained model performance on a selection of data sets. Extracted LONs, while heterogeneous across data sets, all exhibit simple global st...

Find SimilarView on arXiv

September 25, 2023

79% Match

Krzysztof Laddach, Rafał Łangowski, ... , Puchalski Bartosz

A problem related to the development of algorithms designed to find the structure of artificial neural network used for behavioural (black-box) modelling of selected dynamic processes has been addressed in this paper. The research has included four original proposals of algorithms dedicated to neural network architecture search. Algorithms have been based on well-known optimisation techniques such as evolutionary algorithms and gradient descent methods. In the presented resea...

Find SimilarView on arXiv

August 20, 2022

78% Match

Jun Yuan, Mengchen Liu, ... , Liu Shixia

Recent advances in artificial intelligence largely benefit from better neural network architectures. These architectures are a product of a costly process of trial-and-error. To ease this process, we develop ArchExplorer, a visual analysis method for understanding a neural architecture space and summarizing design principles. The key idea behind our method is to make the architecture space explainable by exploiting structural distances between architectures. We formulate the ...

Find SimilarView on arXiv

March 4, 2024

78% Match

Mengfei Ji, Yuchun Chang, ... , Al-Ars Zaid

As machine learning (ML) algorithms get deployed in an ever-increasing number of applications, these algorithms need to achieve better trade-offs between high accuracy, high throughput and low latency. This paper introduces NASH, a novel approach that applies neural architecture search to machine learning hardware. Using NASH, hardware designs can achieve not only high throughput and low latency but also superior accuracy performance. We present four versions of the NASH stra...

Find SimilarView on arXiv