> For the complete documentation index, see [llms.txt](https://julienbeaulieu.gitbook.io/wiki/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://julienbeaulieu.gitbook.io/wiki/sciences/machine-learning/neural-networks.md).

# Neural Networks

![](/files/-LlIzx-OsVwweH7s0YC5)

Each node in the hidden layer adds to the model's ability to capture interactions or abstract patterns. The more nodes we have the more interactions we can capture. 

**Representation learning**

* Deep networks internally build representations of patterns in the data = representation learning
* Partially replace the need for feature engineering
* Subsequent layers build increasingly sophisticated representations fo raw data. 

**Some advantages:**

* The modeler doesn't need to specify the interactions 
* When you train the model, the neural network gets weights that find the relevant patterns  to make better predictions

**Loss function**

* Aggregates eorrors in predictions from many data points into a single number. 
* It's a measure of the model's predictive performance. 
* A common loss function for a regression task = squared error

Where do NNs shine

* Input is high-dimensional discrete or real-valued
* Output is discrete or real valued, or a vector of values
* Possibly noisy data
* Form of target function is unknown
* Human interpretability is not important
* The computation of the output based on the input has to be fast
* (Highly) non-linear models
* Can learn to order/rank inputs easily
* Scale to very large datasets
* Very flexible models
* Composed of simple units (neurons)
* Adapt to different types of data
* May require “fiddling” with model architecture + optimization hyperparameters
* Standardizing data can be very important