Hello, dear friend, you can consult us at any time if you have any questions, add WeChat: daixieit

Problem 1 (5 points)

Goal: Build and train an artifcial neural network for a binary classification task.

You are encouraged to use PyTorch, TensorFlow or the Keras API, but any other deep learning library (in Python, Julia or Matlab)would be acceptable for this homework assignment.

Data: The input data and their corresponding binary labels are provided in the data fle,hw1data.dat ↓. The input data contains  1000 two-dimensional data points that  lie within a square of area one.The input data and labels should be loaded by reading the data fle using any choice of library. Please note that for binary classification, the-1/1 labels need to be converted into 0/1 labels.

Architecture: A typical neural network consists of densely connected layers also called fully connected layers.It means all the inputs are connected to the outputs. Define a Sequential model, wherein the layers are stacked sequentially and each layer has exactly one input tensor and one output tensor. Please build an artificial neural network by adding the following layers to the Sequential model using the configuration below.

Input                       Shape:2

Dense                     Units:5

Dense                    Units:1         Activation:Sigmoid

The initial random weights of layers can be defined by specifying weight and bias initializers. For each of the above layers, initialize the kernel weights from a Xavier/Glorot uniform distribution and set the random seed to 99. Additionally, initialize the bias vector as a zero vector. The activation function defines the node output given a set of inputs. An appropriate choice of activation function is required to allow the artificial neural network to learn a non-linear pattern. The activation functions for the first dense layer can be chosen from some of the commonly used activation functions like Rectifed Linear Unit(ReLU), Hyperbolic Tangent (tanh),and Sigmoid.

Training: The model is compiled by specifying the optimizer, the loss function and metrics to be recorded at each step of the training process.For binary classification,it is a common practice to use binary cross- entropy as loss function. Popular deep learning libraries provide support for several optimization algorithms. Some of them are Stochastic gradient descent (SGD), RMSprop, ADAM. Please choose accuracy as a metric during model compilation. Finally, train the artificial neural network by fitting the input data and labels with each of the aforementioned optimizers and their respective confguration as given in the table below. The neural network should be trained until convergence is achieved.

Deliverables:

·Please report the training accuracy after the training process is carried out for every combination of activation function and optimizer.

Plot the loss curves to determine the number of epochs required to achieve convergence.

Report the hyperparameter tuning step.

Predict and report the binary classifcation results for the data point [0.8,0.2] with the trained artifcial neural network.

·Discuss the influence of particular parameters on different optimizers.

It is recommended that the final results be reported in a tabular format as shown below. Please also make sure to submit your working code files along with the final results.

Optimizer

Activation function

Required epochs

Training accuracy(%)

Prediction for [0.8, 0.2]

SGD

(Learning rate =0.01,Momentum=[0.0,0.1,0.5,0.9], discuss the impact of momentum values on the

convergence behavior of the SGD optimizer)

ReLU

Tanh

Sigmoid

RMSprop

(Learning rate =[0.0001,0.001,0.01],discuss the effect of learning rates on learning curves,Epsilon =10-6)

ReLU

Tanh

Sigmoid

ADAM

(β1=[0.85,0.9],β=[0.95,0.99],discuss the functions of the parameters  β1 and β)

ReLU

Tanh

Sigmoid