Solving ODEs with NeuralPDE.jl

From https://docs.sciml.ai/NeuralPDE/stable/tutorials/ode/

For example, to solve the ODE

\[ u^{\prime} = cos(2 \pi t) \]

using NeuralPDE
using Lux
using OptimizationOptimisers
using OrdinaryDiffEq
using LinearAlgebra
using Random
rng = Random.default_rng()

TaskLocalRNG()

The true function is a cosine wave in this example.

model(u, p, t) = cospi(2t)

tspan = (0.0f0, 1.0f0)
u0 = 0.0f0
prob = ODEProblem(model, u0, tspan)

ODEProblem with uType Float32 and tType Float32. In-place: false
timespan: (0.0f0, 1.0f0)
u0: 0.0f0

Construct a neural network to solve the problem.

chain = Lux.Chain(Lux.Dense(1, 5, σ), Lux.Dense(5, 1))
ps, st = Lux.setup(rng, chain)

((layer_1 = (weight = Float32[-0.6037462; -0.887221; … ; 0.21361578; 0.45948505;;], bias = Float32[0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = Float32[-0.51698124 0.916435 … 0.622723 0.2210002], bias = Float32[0.0;;])), (layer_1 = NamedTuple(), layer_2 = NamedTuple()))

We solve the ODE as before, just change the solver algorithm to NeuralPDE.NNODE().

optimizer = OptimizationOptimisers.Adam(0.1)
alg = NeuralPDE.NNODE(chain, optimizer, init_params = ps)

NNODE{Chain{@NamedTuple{layer_1::Dense{typeof(sigmoid_fast), Int64, Int64, typeof(glorot_uniform), typeof(zeros32), Static.True}, layer_2::Dense{typeof(identity), Int64, Int64, typeof(glorot_uniform), typeof(zeros32), Static.True}}, Nothing}, Adam, Nothing, Bool, Bool, Base.Pairs{Symbol, @NamedTuple{layer_1::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}, layer_2::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}}, Tuple{Symbol}, @NamedTuple{init_params::@NamedTuple{layer_1::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}, layer_2::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}}}}, Nothing, Nothing}(Chain{@NamedTuple{layer_1::Dense{typeof(sigmoid_fast), Int64, Int64, typeof(glorot_uniform), typeof(zeros32), Static.True}, layer_2::Dense{typeof(identity), Int64, Int64, typeof(glorot_uniform), typeof(zeros32), Static.True}}, Nothing}((layer_1 = Dense(1 => 5, sigmoid_fast), layer_2 = Dense(5 => 1)), nothing), Adam(0.1, (0.9, 0.999), 1.0e-8), nothing, false, true, nothing, false, nothing, Base.Pairs{Symbol, @NamedTuple{layer_1::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}, layer_2::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}}, Tuple{Symbol}, @NamedTuple{init_params::@NamedTuple{layer_1::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}, layer_2::@NamedTuple{weight::Matrix{Float32}, bias::Matrix{Float32}}}}}(:init_params => (layer_1 = (weight = [-0.6037462; -0.887221; … ; 0.21361578; 0.45948505;;], bias = [0.0; 0.0; … ; 0.0; 0.0;;]), layer_2 = (weight = [-0.51698124 0.916435 … 0.622723 0.2210002], bias = [0.0;;]))))

sol = solve(prob, alg, verbose=true, maxiters=2000, saveat = 0.01)

Current loss is: 0.69381344, Iteration: 1

Current loss is: 0.44624043, Iteration: 2
Current loss is: 0.6467843, Iteration: 3
Current loss is: 0.48197794, Iteration: 4
Current loss is: 0.366822, Iteration: 5
Current loss is: 0.42887565, Iteration: 6
Current loss is: 0.5023174, Iteration: 7
Current loss is: 0.46324787, Iteration: 8
Current loss is: 0.3871771, Iteration: 9
Current loss is: 0.3671282, Iteration: 10
Current loss is: 0.40472373, Iteration: 11
Current loss is: 0.43672854, Iteration: 12
Current loss is: 0.42266083, Iteration: 13
Current loss is: 0.38383022, Iteration: 14
Current loss is: 0.3626445, Iteration: 15
Current loss is: 0.37470597, Iteration: 16
Current loss is: 0.39726067, Iteration: 17
Current loss is: 0.399786, Iteration: 18
Current loss is: 0.3802567, Iteration: 19
Current loss is: 0.36145264, Iteration: 20
Current loss is: 0.36061856, Iteration: 21
Current loss is: 0.37271065, Iteration: 22
Current loss is: 0.37952662, Iteration: 23
Current loss is: 0.37223035, Iteration: 24
Current loss is: 0.35889718, Iteration: 25
Current loss is: 0.35291174, Iteration: 26
Current loss is: 0.3575132, Iteration: 27
Current loss is: 0.36352545, Iteration: 28
Current loss is: 0.36139357, Iteration: 29
Current loss is: 0.3526975, Iteration: 30
Current loss is: 0.3464644, Iteration: 31
Current loss is: 0.34730214, Iteration: 32
Current loss is: 0.3505348, Iteration: 33
Current loss is: 0.34963948, Iteration: 34
Current loss is: 0.34387004, Iteration: 35
Current loss is: 0.33883214, Iteration: 36
Current loss is: 0.3380912, Iteration: 37
Current loss is: 0.33951694, Iteration: 38
Current loss is: 0.3382343, Iteration: 39
Current loss is: 0.33376455, Iteration: 40
Current loss is: 0.32978696, Iteration: 41
Current loss is: 0.32846215, Iteration: 42
Current loss is: 0.32829627, Iteration: 43
Current loss is: 0.32624573, Iteration: 44
Current loss is: 0.322316, Iteration: 45
Current loss is: 0.31894362, Iteration: 46
Current loss is: 0.31736195, Iteration: 47
Current loss is: 0.3161139, Iteration: 48
Current loss is: 0.31308347, Iteration: 49
Current loss is: 0.30925494, Iteration: 50
Current loss is: 0.30626616, Iteration: 51
Current loss is: 0.30415106, Iteration: 52
Current loss is: 0.3015473, Iteration: 53
Current loss is: 0.29786143, Iteration: 54
Current loss is: 0.29398364, Iteration: 55

Current loss is: 0.29079312, Iteration: 56
Current loss is: 0.2877394, Iteration: 57
Current loss is: 0.2840404, Iteration: 58
Current loss is: 0.2796609, Iteration: 59
Current loss is: 0.27558413, Iteration: 60
Current loss is: 0.27167594, Iteration: 61
Current loss is: 0.26750356, Iteration: 62
Current loss is: 0.26262224, Iteration: 63
Current loss is: 0.25776017, Iteration: 64
Current loss is: 0.25301537, Iteration: 65
Current loss is: 0.24810168, Iteration: 66
Current loss is: 0.24268061, Iteration: 67
Current loss is: 0.23708001, Iteration: 68
Current loss is: 0.2314705, Iteration: 69
Current loss is: 0.22576217, Iteration: 70
Current loss is: 0.21969646, Iteration: 71
Current loss is: 0.21336499, Iteration: 72
Current loss is: 0.20701098, Iteration: 73
Current loss is: 0.2005964, Iteration: 74
Current loss is: 0.19405498, Iteration: 75
Current loss is: 0.1871837, Iteration: 76
Current loss is: 0.18027964, Iteration: 77
Current loss is: 0.17338832, Iteration: 78
Current loss is: 0.16637921, Iteration: 79
Current loss is: 0.15928406, Iteration: 80
Current loss is: 0.15215997, Iteration: 81
Current loss is: 0.14499816, Iteration: 82
Current loss is: 0.13792616, Iteration: 83
Current loss is: 0.13097163, Iteration: 84
Current loss is: 0.12387872, Iteration: 85
Current loss is: 0.11699815, Iteration: 86
Current loss is: 0.110282965, Iteration: 87
Current loss is: 0.103666216, Iteration: 88
Current loss is: 0.09721505, Iteration: 89
Current loss is: 0.09088342, Iteration: 90
Current loss is: 0.084858954, Iteration: 91
Current loss is: 0.07906141, Iteration: 92
Current loss is: 0.07341853, Iteration: 93
Current loss is: 0.068068, Iteration: 94
Current loss is: 0.06293434, Iteration: 95
Current loss is: 0.05807048, Iteration: 96
Current loss is: 0.05355919, Iteration: 97
Current loss is: 0.04926432, Iteration: 98
Current loss is: 0.04519561, Iteration: 99
Current loss is: 0.041519884, Iteration: 100
Current loss is: 0.037972417, Iteration: 101
Current loss is: 0.034736995, Iteration: 102
Current loss is: 0.031712636, Iteration: 103
Current loss is: 0.028926272, Iteration: 104
Current loss is: 0.026411153, Iteration: 105
Current loss is: 0.024033088, Iteration: 106
Current loss is: 0.021912288, Iteration: 107
Current loss is: 0.019983914, Iteration: 108
Current loss is: 0.018205922, Iteration: 109
Current loss is: 0.016568676, Iteration: 110
Current loss is: 0.015115879, Iteration: 111
Current loss is: 0.013791087, Iteration: 112
Current loss is: 0.0125702, Iteration: 113
Current loss is: 0.01147961, Iteration: 114
Current loss is: 0.010501751, Iteration: 115
Current loss is: 0.009591864, Iteration: 116
Current loss is: 0.008799787, Iteration: 117
Current loss is: 0.008067986, Iteration: 118
Current loss is: 0.007413298, Iteration: 119
Current loss is: 0.0068251835, Iteration: 120
Current loss is: 0.006286963, Iteration: 121
Current loss is: 0.0058026956, Iteration: 122
Current loss is: 0.0053514102, Iteration: 123
Current loss is: 0.0049520656, Iteration: 124
Current loss is: 0.004604561, Iteration: 125
Current loss is: 0.004275314, Iteration: 126
Current loss is: 0.0039907116, Iteration: 127
Current loss is: 0.0037063884, Iteration: 128
Current loss is: 0.0034679908, Iteration: 129
Current loss is: 0.0032445774, Iteration: 130
Current loss is: 0.0030396578, Iteration: 131
Current loss is: 0.0028576972, Iteration: 132

Current loss is: 0.0026854172, Iteration: 133
Current loss is: 0.00252565, Iteration: 134
Current loss is: 0.002389948, Iteration: 135
Current loss is: 0.0022566435, Iteration: 136
Current loss is: 0.0021345702, Iteration: 137
Current loss is: 0.0020286227, Iteration: 138
Current loss is: 0.0019286035, Iteration: 139
Current loss is: 0.0018288189, Iteration: 140
Current loss is: 0.001741783, Iteration: 141
Current loss is: 0.001662984, Iteration: 142
Current loss is: 0.0015846739, Iteration: 143
Current loss is: 0.0015178228, Iteration: 144
Current loss is: 0.0014508306, Iteration: 145
Current loss is: 0.0013921348, Iteration: 146
Current loss is: 0.0013392138, Iteration: 147
Current loss is: 0.0012842247, Iteration: 148
Current loss is: 0.0012326632, Iteration: 149
Current loss is: 0.001186714, Iteration: 150
Current loss is: 0.0011440378, Iteration: 151
Current loss is: 0.0011029968, Iteration: 152
Current loss is: 0.0010644407, Iteration: 153
Current loss is: 0.0010258663, Iteration: 154
Current loss is: 0.0009938087, Iteration: 155
Current loss is: 0.0009612789, Iteration: 156
Current loss is: 0.0009302119, Iteration: 157
Current loss is: 0.00090267975, Iteration: 158
Current loss is: 0.00087341905, Iteration: 159
Current loss is: 0.0008474229, Iteration: 160
Current loss is: 0.0008248752, Iteration: 161
Current loss is: 0.0008003466, Iteration: 162
Current loss is: 0.0007783482, Iteration: 163
Current loss is: 0.0007540134, Iteration: 164
Current loss is: 0.0007352028, Iteration: 165
Current loss is: 0.00071584585, Iteration: 166
Current loss is: 0.0006973833, Iteration: 167
Current loss is: 0.00067943666, Iteration: 168
Current loss is: 0.00066307443, Iteration: 169
Current loss is: 0.00064553344, Iteration: 170
Current loss is: 0.00062949467, Iteration: 171
Current loss is: 0.00061465084, Iteration: 172
Current loss is: 0.00060023245, Iteration: 173
Current loss is: 0.00058697327, Iteration: 174
Current loss is: 0.0005723901, Iteration: 175
Current loss is: 0.000559455, Iteration: 176
Current loss is: 0.00054803636, Iteration: 177
Current loss is: 0.0005364419, Iteration: 178
Current loss is: 0.0005243468, Iteration: 179
Current loss is: 0.00051348563, Iteration: 180
Current loss is: 0.0005020919, Iteration: 181
Current loss is: 0.000491556, Iteration: 182
Current loss is: 0.00048114365, Iteration: 183

Current loss is: 0.00047156186, Iteration: 184
Current loss is: 0.00046126873, Iteration: 185
Current loss is: 0.000452654, Iteration: 186
Current loss is: 0.0004444879, Iteration: 187
Current loss is: 0.0004353898, Iteration: 188
Current loss is: 0.0004263841, Iteration: 189
Current loss is: 0.00041655893, Iteration: 190
Current loss is: 0.00040874118, Iteration: 191
Current loss is: 0.00040384382, Iteration: 192
Current loss is: 0.00039522204, Iteration: 193
Current loss is: 0.00038834562, Iteration: 194
Current loss is: 0.000381379, Iteration: 195
Current loss is: 0.0003747336, Iteration: 196
Current loss is: 0.000366949, Iteration: 197
Current loss is: 0.00036147423, Iteration: 198
Current loss is: 0.00035525206, Iteration: 199
Current loss is: 0.00034842774, Iteration: 200
Current loss is: 0.00034324804, Iteration: 201
Current loss is: 0.00033652358, Iteration: 202
Current loss is: 0.0003324102, Iteration: 203
Current loss is: 0.0003257938, Iteration: 204
Current loss is: 0.00032010613, Iteration: 205
Current loss is: 0.0003151634, Iteration: 206
Current loss is: 0.00030939063, Iteration: 207
Current loss is: 0.0003047596, Iteration: 208
Current loss is: 0.00030033445, Iteration: 209
Current loss is: 0.00029689912, Iteration: 210
Current loss is: 0.00029091223, Iteration: 211
Current loss is: 0.00028671714, Iteration: 212
Current loss is: 0.0002817224, Iteration: 213
Current loss is: 0.0002780535, Iteration: 214
Current loss is: 0.00027280295, Iteration: 215
Current loss is: 0.0002691487, Iteration: 216
Current loss is: 0.00026540848, Iteration: 217
Current loss is: 0.00026155048, Iteration: 218
Current loss is: 0.00025802007, Iteration: 219
Current loss is: 0.00025317102, Iteration: 220
Current loss is: 0.00024899433, Iteration: 221
Current loss is: 0.00024598086, Iteration: 222
Current loss is: 0.00024329645, Iteration: 223
Current loss is: 0.00023927962, Iteration: 224
Current loss is: 0.000236037, Iteration: 225
Current loss is: 0.00023220191, Iteration: 226
Current loss is: 0.00022876906, Iteration: 227
Current loss is: 0.00022626435, Iteration: 228
Current loss is: 0.00022301279, Iteration: 229
Current loss is: 0.0002200031, Iteration: 230
Current loss is: 0.00021619463, Iteration: 231
Current loss is: 0.00021358597, Iteration: 232
Current loss is: 0.00021197031, Iteration: 233
Current loss is: 0.0002091907, Iteration: 234
Current loss is: 0.00020570066, Iteration: 235
Current loss is: 0.00020240134, Iteration: 236
Current loss is: 0.0002002295, Iteration: 237
Current loss is: 0.00019841956, Iteration: 238
Current loss is: 0.00019542231, Iteration: 239
Current loss is: 0.00019314568, Iteration: 240
Current loss is: 0.00019023233, Iteration: 241
Current loss is: 0.00018744085, Iteration: 242

Current loss is: 0.00018491253, Iteration: 243
Current loss is: 0.00018296126, Iteration: 244
Current loss is: 0.00018150602, Iteration: 245
Current loss is: 0.00017842339, Iteration: 246
Current loss is: 0.00017586398, Iteration: 247
Current loss is: 0.00017405886, Iteration: 248
Current loss is: 0.00017186669, Iteration: 249
Current loss is: 0.00017059366, Iteration: 250
Current loss is: 0.00016808226, Iteration: 251
Current loss is: 0.00016668448, Iteration: 252
Current loss is: 0.00016431353, Iteration: 253
Current loss is: 0.00016197407, Iteration: 254
Current loss is: 0.00016069318, Iteration: 255
Current loss is: 0.00015811589, Iteration: 256
Current loss is: 0.00015637289, Iteration: 257
Current loss is: 0.00015504542, Iteration: 258
Current loss is: 0.00015303183, Iteration: 259
Current loss is: 0.00015178518, Iteration: 260
Current loss is: 0.00014957212, Iteration: 261
Current loss is: 0.0001483538, Iteration: 262
Current loss is: 0.00014633873, Iteration: 263
Current loss is: 0.00014476472, Iteration: 264
Current loss is: 0.00014328501, Iteration: 265
Current loss is: 0.00014193167, Iteration: 266
Current loss is: 0.00014022602, Iteration: 267
Current loss is: 0.00013862169, Iteration: 268
Current loss is: 0.00013716654, Iteration: 269
Current loss is: 0.00013598015, Iteration: 270
Current loss is: 0.00013402865, Iteration: 271
Current loss is: 0.00013319992, Iteration: 272
Current loss is: 0.00013170177, Iteration: 273
Current loss is: 0.00013050703, Iteration: 274
Current loss is: 0.00012811854, Iteration: 275
Current loss is: 0.0001278967, Iteration: 276
Current loss is: 0.00012628769, Iteration: 277
Current loss is: 0.00012492723, Iteration: 278
Current loss is: 0.00012336021, Iteration: 279
Current loss is: 0.00012221321, Iteration: 280
Current loss is: 0.00012085013, Iteration: 281
Current loss is: 0.00012077288, Iteration: 282
Current loss is: 0.00011869882, Iteration: 283
Current loss is: 0.0001174387, Iteration: 284
Current loss is: 0.0001170795, Iteration: 285
Current loss is: 0.00011548173, Iteration: 286
Current loss is: 0.0001141532, Iteration: 287
Current loss is: 0.0001132594, Iteration: 288
Current loss is: 0.000111820176, Iteration: 289
Current loss is: 0.00011145446, Iteration: 290
Current loss is: 0.00010987962, Iteration: 291
Current loss is: 0.00010867668, Iteration: 292
Current loss is: 0.00010846567, Iteration: 293
Current loss is: 0.000107029286, Iteration: 294
Current loss is: 0.000105542116, Iteration: 295
Current loss is: 0.000104759485, Iteration: 296
Current loss is: 0.00010376782, Iteration: 297
Current loss is: 0.0001032756, Iteration: 298
Current loss is: 0.00010228284, Iteration: 299
Current loss is: 0.00010114994, Iteration: 300
Current loss is: 0.00010050251, Iteration: 301
Current loss is: 9.92832e-5, Iteration: 302
Current loss is: 9.9002704e-5, Iteration: 303
Current loss is: 9.71288e-5, Iteration: 304
Current loss is: 9.663958e-5, Iteration: 305
Current loss is: 9.647731e-5, Iteration: 306
Current loss is: 9.522082e-5, Iteration: 307
Current loss is: 9.493202e-5, Iteration: 308
Current loss is: 9.3274226e-5, Iteration: 309
Current loss is: 9.256136e-5, Iteration: 310
Current loss is: 9.19181e-5, Iteration: 311

Current loss is: 9.1304e-5, Iteration: 312
Current loss is: 9.0193644e-5, Iteration: 313
Current loss is: 8.898828e-5, Iteration: 314
Current loss is: 8.917229e-5, Iteration: 315
Current loss is: 8.793075e-5, Iteration: 316
Current loss is: 8.716122e-5, Iteration: 317
Current loss is: 8.65249e-5, Iteration: 318
Current loss is: 8.596146e-5, Iteration: 319
Current loss is: 8.5172025e-5, Iteration: 320
Current loss is: 8.447323e-5, Iteration: 321
Current loss is: 8.3820945e-5, Iteration: 322
Current loss is: 8.287176e-5, Iteration: 323
Current loss is: 8.245516e-5, Iteration: 324
Current loss is: 8.209896e-5, Iteration: 325
Current loss is: 8.1309474e-5, Iteration: 326
Current loss is: 8.027661e-5, Iteration: 327
Current loss is: 7.981867e-5, Iteration: 328
Current loss is: 7.970496e-5, Iteration: 329
Current loss is: 7.8846206e-5, Iteration: 330
Current loss is: 7.807376e-5, Iteration: 331
Current loss is: 7.7851604e-5, Iteration: 332
Current loss is: 7.709749e-5, Iteration: 333
Current loss is: 7.636416e-5, Iteration: 334
Current loss is: 7.5670425e-5, Iteration: 335
Current loss is: 7.525351e-5, Iteration: 336
Current loss is: 7.460981e-5, Iteration: 337
Current loss is: 7.473647e-5, Iteration: 338
Current loss is: 7.389151e-5, Iteration: 339
Current loss is: 7.302566e-5, Iteration: 340
Current loss is: 7.250512e-5, Iteration: 341
Current loss is: 7.163947e-5, Iteration: 342
Current loss is: 7.122416e-5, Iteration: 343
Current loss is: 7.083464e-5, Iteration: 344
Current loss is: 7.0441936e-5, Iteration: 345
Current loss is: 6.971404e-5, Iteration: 346
Current loss is: 6.919232e-5, Iteration: 347
Current loss is: 6.87104e-5, Iteration: 348
Current loss is: 6.820334e-5, Iteration: 349
Current loss is: 6.8049216e-5, Iteration: 350
Current loss is: 6.744534e-5, Iteration: 351
Current loss is: 6.734698e-5, Iteration: 352
Current loss is: 6.6457455e-5, Iteration: 353
Current loss is: 6.6316876e-5, Iteration: 354
Current loss is: 6.582397e-5, Iteration: 355
Current loss is: 6.519518e-5, Iteration: 356
Current loss is: 6.452029e-5, Iteration: 357
Current loss is: 6.4092266e-5, Iteration: 358
Current loss is: 6.4132306e-5, Iteration: 359
Current loss is: 6.306143e-5, Iteration: 360
Current loss is: 6.319652e-5, Iteration: 361
Current loss is: 6.23782e-5, Iteration: 362
Current loss is: 6.204874e-5, Iteration: 363
Current loss is: 6.188623e-5, Iteration: 364
Current loss is: 6.1190614e-5, Iteration: 365
Current loss is: 6.0977123e-5, Iteration: 366
Current loss is: 6.0448903e-5, Iteration: 367
Current loss is: 6.0329916e-5, Iteration: 368
Current loss is: 5.966232e-5, Iteration: 369
Current loss is: 5.9486094e-5, Iteration: 370
Current loss is: 5.855911e-5, Iteration: 371
Current loss is: 5.8561054e-5, Iteration: 372
Current loss is: 5.7907284e-5, Iteration: 373
Current loss is: 5.776079e-5, Iteration: 374
Current loss is: 5.723638e-5, Iteration: 375
Current loss is: 5.6952624e-5, Iteration: 376
Current loss is: 5.669119e-5, Iteration: 377
Current loss is: 5.655905e-5, Iteration: 378
Current loss is: 5.606123e-5, Iteration: 379
Current loss is: 5.526714e-5, Iteration: 380
Current loss is: 5.5011544e-5, Iteration: 381
Current loss is: 5.5154414e-5, Iteration: 382
Current loss is: 5.4349923e-5, Iteration: 383
Current loss is: 5.4093107e-5, Iteration: 384
Current loss is: 5.38019e-5, Iteration: 385
Current loss is: 5.322461e-5, Iteration: 386
Current loss is: 5.3344727e-5, Iteration: 387
Current loss is: 5.3215896e-5, Iteration: 388
Current loss is: 5.3031014e-5, Iteration: 389
Current loss is: 5.224279e-5, Iteration: 390
Current loss is: 5.191232e-5, Iteration: 391
Current loss is: 5.1486928e-5, Iteration: 392
Current loss is: 5.0999e-5, Iteration: 393
Current loss is: 5.0970266e-5, Iteration: 394
Current loss is: 5.068741e-5, Iteration: 395
Current loss is: 5.074697e-5, Iteration: 396
Current loss is: 4.99071e-5, Iteration: 397
Current loss is: 4.9812574e-5, Iteration: 398
Current loss is: 4.968775e-5, Iteration: 399
Current loss is: 4.9453036e-5, Iteration: 400
Current loss is: 4.9210645e-5, Iteration: 401
Current loss is: 4.8488484e-5, Iteration: 402
Current loss is: 4.823707e-5, Iteration: 403
Current loss is: 4.7956735e-5, Iteration: 404
Current loss is: 4.7560665e-5, Iteration: 405

Current loss is: 4.7172944e-5, Iteration: 406
Current loss is: 4.7284564e-5, Iteration: 407
Current loss is: 4.6861893e-5, Iteration: 408
Current loss is: 4.6479236e-5, Iteration: 409
Current loss is: 4.659966e-5, Iteration: 410
Current loss is: 4.638027e-5, Iteration: 411
Current loss is: 4.6245237e-5, Iteration: 412
Current loss is: 4.5899043e-5, Iteration: 413
Current loss is: 4.541986e-5, Iteration: 414
Current loss is: 4.5141693e-5, Iteration: 415
Current loss is: 4.4852928e-5, Iteration: 416
Current loss is: 4.4585606e-5, Iteration: 417
Current loss is: 4.4778342e-5, Iteration: 418
Current loss is: 4.446815e-5, Iteration: 419
Current loss is: 4.4239147e-5, Iteration: 420
Current loss is: 4.3431024e-5, Iteration: 421
Current loss is: 4.3397013e-5, Iteration: 422
Current loss is: 4.362511e-5, Iteration: 423
Current loss is: 4.326466e-5, Iteration: 424
Current loss is: 4.2725653e-5, Iteration: 425
Current loss is: 4.282134e-5, Iteration: 426
Current loss is: 4.2593845e-5, Iteration: 427
Current loss is: 4.2192154e-5, Iteration: 428
Current loss is: 4.2019416e-5, Iteration: 429
Current loss is: 4.1576437e-5, Iteration: 430
Current loss is: 4.1277606e-5, Iteration: 431
Current loss is: 4.1541934e-5, Iteration: 432
Current loss is: 4.121098e-5, Iteration: 433
Current loss is: 4.093694e-5, Iteration: 434
Current loss is: 4.091839e-5, Iteration: 435
Current loss is: 4.0219547e-5, Iteration: 436
Current loss is: 4.017771e-5, Iteration: 437
Current loss is: 4.0047293e-5, Iteration: 438
Current loss is: 3.9720933e-5, Iteration: 439
Current loss is: 3.9736376e-5, Iteration: 440
Current loss is: 3.933361e-5, Iteration: 441
Current loss is: 3.9218958e-5, Iteration: 442
Current loss is: 3.9219463e-5, Iteration: 443
Current loss is: 3.909162e-5, Iteration: 444
Current loss is: 3.8473005e-5, Iteration: 445
Current loss is: 3.874127e-5, Iteration: 446
Current loss is: 3.847103e-5, Iteration: 447
Current loss is: 3.8018003e-5, Iteration: 448
Current loss is: 3.788326e-5, Iteration: 449
Current loss is: 3.7827544e-5, Iteration: 450
Current loss is: 3.757854e-5, Iteration: 451
Current loss is: 3.7510243e-5, Iteration: 452
Current loss is: 3.7089278e-5, Iteration: 453
Current loss is: 3.708055e-5, Iteration: 454
Current loss is: 3.6712398e-5, Iteration: 455
Current loss is: 3.6780868e-5, Iteration: 456
Current loss is: 3.6383648e-5, Iteration: 457
Current loss is: 3.6276826e-5, Iteration: 458
Current loss is: 3.624897e-5, Iteration: 459
Current loss is: 3.5730016e-5, Iteration: 460
Current loss is: 3.5642573e-5, Iteration: 461
Current loss is: 3.5624405e-5, Iteration: 462
Current loss is: 3.553302e-5, Iteration: 463
Current loss is: 3.5352066e-5, Iteration: 464
Current loss is: 3.544793e-5, Iteration: 465
Current loss is: 3.458873e-5, Iteration: 466
Current loss is: 3.4464516e-5, Iteration: 467
Current loss is: 3.453354e-5, Iteration: 468
Current loss is: 3.4749042e-5, Iteration: 469
Current loss is: 3.405634e-5, Iteration: 470
Current loss is: 3.4174824e-5, Iteration: 471
Current loss is: 3.3709628e-5, Iteration: 472
Current loss is: 3.3653792e-5, Iteration: 473
Current loss is: 3.3536875e-5, Iteration: 474
Current loss is: 3.349127e-5, Iteration: 475
Current loss is: 3.321204e-5, Iteration: 476
Current loss is: 3.309626e-5, Iteration: 477
Current loss is: 3.317981e-5, Iteration: 478
Current loss is: 3.278641e-5, Iteration: 479
Current loss is: 3.2821958e-5, Iteration: 480
Current loss is: 3.2760723e-5, Iteration: 481
Current loss is: 3.249244e-5, Iteration: 482
Current loss is: 3.2223346e-5, Iteration: 483
Current loss is: 3.2314412e-5, Iteration: 484
Current loss is: 3.2038428e-5, Iteration: 485
Current loss is: 3.1563788e-5, Iteration: 486
Current loss is: 3.2056792e-5, Iteration: 487
Current loss is: 3.1334217e-5, Iteration: 488
Current loss is: 3.1470416e-5, Iteration: 489
Current loss is: 3.0931777e-5, Iteration: 490
Current loss is: 3.1096588e-5, Iteration: 491
Current loss is: 3.1180494e-5, Iteration: 492
Current loss is: 3.091997e-5, Iteration: 493
Current loss is: 3.070304e-5, Iteration: 494
Current loss is: 3.0491165e-5, Iteration: 495
Current loss is: 3.0273604e-5, Iteration: 496
Current loss is: 3.068945e-5, Iteration: 497
Current loss is: 3.0496045e-5, Iteration: 498
Current loss is: 3.012093e-5, Iteration: 499
Current loss is: 3.0246565e-5, Iteration: 500
Current loss is: 2.9635923e-5, Iteration: 501
Current loss is: 2.9760691e-5, Iteration: 502
Current loss is: 2.9608129e-5, Iteration: 503
Current loss is: 2.9764296e-5, Iteration: 504
Current loss is: 2.9579369e-5, Iteration: 505
Current loss is: 2.9427778e-5, Iteration: 506
Current loss is: 2.8916791e-5, Iteration: 507
Current loss is: 2.8768232e-5, Iteration: 508
Current loss is: 2.8766754e-5, Iteration: 509
Current loss is: 2.8734219e-5, Iteration: 510
Current loss is: 2.8497023e-5, Iteration: 511
Current loss is: 2.8277016e-5, Iteration: 512
Current loss is: 2.8496921e-5, Iteration: 513
Current loss is: 2.8268169e-5, Iteration: 514
Current loss is: 2.8554317e-5, Iteration: 515
Current loss is: 2.796709e-5, Iteration: 516
Current loss is: 2.7957853e-5, Iteration: 517
Current loss is: 2.7925984e-5, Iteration: 518
Current loss is: 2.760133e-5, Iteration: 519
Current loss is: 2.787037e-5, Iteration: 520
Current loss is: 2.7175765e-5, Iteration: 521
Current loss is: 2.734482e-5, Iteration: 522
Current loss is: 2.6918027e-5, Iteration: 523
Current loss is: 2.7295438e-5, Iteration: 524
Current loss is: 2.7038766e-5, Iteration: 525
Current loss is: 2.7175014e-5, Iteration: 526
Current loss is: 2.6857793e-5, Iteration: 527
Current loss is: 2.6749874e-5, Iteration: 528
Current loss is: 2.6683896e-5, Iteration: 529
Current loss is: 2.6563248e-5, Iteration: 530
Current loss is: 2.6174786e-5, Iteration: 531
Current loss is: 2.63574e-5, Iteration: 532
Current loss is: 2.6150039e-5, Iteration: 533
Current loss is: 2.6058944e-5, Iteration: 534
Current loss is: 2.6011548e-5, Iteration: 535
Current loss is: 2.596925e-5, Iteration: 536
Current loss is: 2.555284e-5, Iteration: 537
Current loss is: 2.565092e-5, Iteration: 538
Current loss is: 2.5783203e-5, Iteration: 539
Current loss is: 2.5777086e-5, Iteration: 540
Current loss is: 2.5473968e-5, Iteration: 541
Current loss is: 2.5039026e-5, Iteration: 542
Current loss is: 2.5240575e-5, Iteration: 543
Current loss is: 2.5256722e-5, Iteration: 544
Current loss is: 2.5030411e-5, Iteration: 545
Current loss is: 2.5178237e-5, Iteration: 546
Current loss is: 2.5005913e-5, Iteration: 547
Current loss is: 2.4735393e-5, Iteration: 548
Current loss is: 2.455154e-5, Iteration: 549
Current loss is: 2.435809e-5, Iteration: 550
Current loss is: 2.463424e-5, Iteration: 551
Current loss is: 2.4379746e-5, Iteration: 552
Current loss is: 2.4320294e-5, Iteration: 553
Current loss is: 2.4201436e-5, Iteration: 554
Current loss is: 2.3964e-5, Iteration: 555
Current loss is: 2.4272826e-5, Iteration: 556
Current loss is: 2.3916673e-5, Iteration: 557
Current loss is: 2.3705601e-5, Iteration: 558
Current loss is: 2.362128e-5, Iteration: 559
Current loss is: 2.3596373e-5, Iteration: 560
Current loss is: 2.3491595e-5, Iteration: 561
Current loss is: 2.3723762e-5, Iteration: 562
Current loss is: 2.3275457e-5, Iteration: 563
Current loss is: 2.3313673e-5, Iteration: 564
Current loss is: 2.3164468e-5, Iteration: 565
Current loss is: 2.3019866e-5, Iteration: 566
Current loss is: 2.3033454e-5, Iteration: 567
Current loss is: 2.298802e-5, Iteration: 568
Current loss is: 2.3196944e-5, Iteration: 569
Current loss is: 2.2663622e-5, Iteration: 570
Current loss is: 2.2616474e-5, Iteration: 571
Current loss is: 2.2685796e-5, Iteration: 572
Current loss is: 2.2458464e-5, Iteration: 573
Current loss is: 2.2439435e-5, Iteration: 574
Current loss is: 2.2507982e-5, Iteration: 575
Current loss is: 2.2398717e-5, Iteration: 576
Current loss is: 2.1976099e-5, Iteration: 577
Current loss is: 2.2484117e-5, Iteration: 578
Current loss is: 2.1961816e-5, Iteration: 579
Current loss is: 2.2128334e-5, Iteration: 580
Current loss is: 2.2131946e-5, Iteration: 581
Current loss is: 2.2160672e-5, Iteration: 582
Current loss is: 2.1645945e-5, Iteration: 583
Current loss is: 2.183234e-5, Iteration: 584
Current loss is: 2.1892243e-5, Iteration: 585

Current loss is: 2.162197e-5, Iteration: 586
Current loss is: 2.1412488e-5, Iteration: 587
Current loss is: 2.1671145e-5, Iteration: 588
Current loss is: 2.1164531e-5, Iteration: 589
Current loss is: 2.1225784e-5, Iteration: 590
Current loss is: 2.1345695e-5, Iteration: 591
Current loss is: 2.123642e-5, Iteration: 592
Current loss is: 2.1315765e-5, Iteration: 593
Current loss is: 2.114564e-5, Iteration: 594
Current loss is: 2.0885842e-5, Iteration: 595
Current loss is: 2.097151e-5, Iteration: 596
Current loss is: 2.0741269e-5, Iteration: 597
Current loss is: 2.090985e-5, Iteration: 598
Current loss is: 2.0356161e-5, Iteration: 599
Current loss is: 2.0792877e-5, Iteration: 600
Current loss is: 2.0619138e-5, Iteration: 601
Current loss is: 2.0370053e-5, Iteration: 602
Current loss is: 2.0276886e-5, Iteration: 603
Current loss is: 2.0422889e-5, Iteration: 604
Current loss is: 2.039922e-5, Iteration: 605
Current loss is: 2.028388e-5, Iteration: 606
Current loss is: 2.0201935e-5, Iteration: 607
Current loss is: 1.9962048e-5, Iteration: 608
Current loss is: 2.007212e-5, Iteration: 609
Current loss is: 1.9972396e-5, Iteration: 610
Current loss is: 1.9904028e-5, Iteration: 611
Current loss is: 1.973814e-5, Iteration: 612
Current loss is: 1.9803076e-5, Iteration: 613
Current loss is: 1.9613655e-5, Iteration: 614
Current loss is: 1.9533461e-5, Iteration: 615
Current loss is: 1.9305302e-5, Iteration: 616
Current loss is: 1.9277091e-5, Iteration: 617
Current loss is: 1.943116e-5, Iteration: 618
Current loss is: 1.9589024e-5, Iteration: 619
Current loss is: 1.920006e-5, Iteration: 620
Current loss is: 1.9108744e-5, Iteration: 621
Current loss is: 1.9217823e-5, Iteration: 622
Current loss is: 1.9385718e-5, Iteration: 623
Current loss is: 1.9000487e-5, Iteration: 624
Current loss is: 1.9138373e-5, Iteration: 625
Current loss is: 1.9044608e-5, Iteration: 626
Current loss is: 1.890474e-5, Iteration: 627
Current loss is: 1.896856e-5, Iteration: 628
Current loss is: 1.8719466e-5, Iteration: 629
Current loss is: 1.8589868e-5, Iteration: 630
Current loss is: 1.8881652e-5, Iteration: 631
Current loss is: 1.8499823e-5, Iteration: 632
Current loss is: 1.8650648e-5, Iteration: 633
Current loss is: 1.8473356e-5, Iteration: 634
Current loss is: 1.8329141e-5, Iteration: 635
Current loss is: 1.8214905e-5, Iteration: 636
Current loss is: 1.8349667e-5, Iteration: 637
Current loss is: 1.8441748e-5, Iteration: 638
Current loss is: 1.8030463e-5, Iteration: 639
Current loss is: 1.8126926e-5, Iteration: 640
Current loss is: 1.8021594e-5, Iteration: 641
Current loss is: 1.7899802e-5, Iteration: 642
Current loss is: 1.823848e-5, Iteration: 643
Current loss is: 1.7765711e-5, Iteration: 644
Current loss is: 1.7759976e-5, Iteration: 645
Current loss is: 1.7821114e-5, Iteration: 646
Current loss is: 1.778763e-5, Iteration: 647
Current loss is: 1.774513e-5, Iteration: 648
Current loss is: 1.7676173e-5, Iteration: 649
Current loss is: 1.7529746e-5, Iteration: 650
Current loss is: 1.7526629e-5, Iteration: 651
Current loss is: 1.756782e-5, Iteration: 652
Current loss is: 1.7359955e-5, Iteration: 653
Current loss is: 1.7299582e-5, Iteration: 654
Current loss is: 1.7579452e-5, Iteration: 655
Current loss is: 1.717272e-5, Iteration: 656
Current loss is: 1.7116707e-5, Iteration: 657
Current loss is: 1.714003e-5, Iteration: 658
Current loss is: 1.6905931e-5, Iteration: 659
Current loss is: 1.6846227e-5, Iteration: 660
Current loss is: 1.7004913e-5, Iteration: 661
Current loss is: 1.6866446e-5, Iteration: 662
Current loss is: 1.6934762e-5, Iteration: 663
Current loss is: 1.6724474e-5, Iteration: 664
Current loss is: 1.6930746e-5, Iteration: 665
Current loss is: 1.6750866e-5, Iteration: 666
Current loss is: 1.6655664e-5, Iteration: 667
Current loss is: 1.6649852e-5, Iteration: 668
Current loss is: 1.68112e-5, Iteration: 669
Current loss is: 1.6591655e-5, Iteration: 670
Current loss is: 1.6427754e-5, Iteration: 671
Current loss is: 1.6405407e-5, Iteration: 672
Current loss is: 1.6913475e-5, Iteration: 673
Current loss is: 1.6239288e-5, Iteration: 674
Current loss is: 1.6393542e-5, Iteration: 675
Current loss is: 1.6394537e-5, Iteration: 676
Current loss is: 1.629675e-5, Iteration: 677
Current loss is: 1.6156702e-5, Iteration: 678
Current loss is: 1.605479e-5, Iteration: 679
Current loss is: 1.5989486e-5, Iteration: 680
Current loss is: 1.5888005e-5, Iteration: 681
Current loss is: 1.6033548e-5, Iteration: 682
Current loss is: 1.5978492e-5, Iteration: 683
Current loss is: 1.5844e-5, Iteration: 684
Current loss is: 1.5895614e-5, Iteration: 685
Current loss is: 1.5944726e-5, Iteration: 686
Current loss is: 1.5906897e-5, Iteration: 687
Current loss is: 1.6092841e-5, Iteration: 688
Current loss is: 1.573214e-5, Iteration: 689
Current loss is: 1.605499e-5, Iteration: 690
Current loss is: 1.5914651e-5, Iteration: 691
Current loss is: 1.5628368e-5, Iteration: 692
Current loss is: 1.5595147e-5, Iteration: 693
Current loss is: 1.5464266e-5, Iteration: 694
Current loss is: 1.5522397e-5, Iteration: 695
Current loss is: 1.5406567e-5, Iteration: 696
Current loss is: 1.5279691e-5, Iteration: 697
Current loss is: 1.5447476e-5, Iteration: 698
Current loss is: 1.5249054e-5, Iteration: 699
Current loss is: 1.537638e-5, Iteration: 700
Current loss is: 1.5233116e-5, Iteration: 701
Current loss is: 1.5201815e-5, Iteration: 702
Current loss is: 1.5154623e-5, Iteration: 703
Current loss is: 1.5125121e-5, Iteration: 704
Current loss is: 1.499788e-5, Iteration: 705
Current loss is: 1.4945832e-5, Iteration: 706
Current loss is: 1.5317357e-5, Iteration: 707
Current loss is: 1.5005089e-5, Iteration: 708
Current loss is: 1.4684816e-5, Iteration: 709
Current loss is: 1.4802415e-5, Iteration: 710
Current loss is: 1.4743404e-5, Iteration: 711
Current loss is: 1.4990527e-5, Iteration: 712
Current loss is: 1.4857723e-5, Iteration: 713
Current loss is: 1.4771926e-5, Iteration: 714
Current loss is: 1.47851615e-5, Iteration: 715
Current loss is: 1.4773732e-5, Iteration: 716
Current loss is: 1.4675304e-5, Iteration: 717
Current loss is: 1.459105e-5, Iteration: 718
Current loss is: 1.4465313e-5, Iteration: 719
Current loss is: 1.4379539e-5, Iteration: 720
Current loss is: 1.427268e-5, Iteration: 721
Current loss is: 1.4339441e-5, Iteration: 722
Current loss is: 1.4447583e-5, Iteration: 723
Current loss is: 1.4366604e-5, Iteration: 724
Current loss is: 1.4360998e-5, Iteration: 725
Current loss is: 1.420675e-5, Iteration: 726
Current loss is: 1.41336905e-5, Iteration: 727
Current loss is: 1.4416533e-5, Iteration: 728
Current loss is: 1.4056701e-5, Iteration: 729
Current loss is: 1.4115713e-5, Iteration: 730
Current loss is: 1.4028011e-5, Iteration: 731
Current loss is: 1.4202761e-5, Iteration: 732
Current loss is: 1.3936327e-5, Iteration: 733
Current loss is: 1.3825832e-5, Iteration: 734
Current loss is: 1.39638405e-5, Iteration: 735
Current loss is: 1.3905631e-5, Iteration: 736
Current loss is: 1.3949245e-5, Iteration: 737
Current loss is: 1.3812905e-5, Iteration: 738
Current loss is: 1.3859086e-5, Iteration: 739
Current loss is: 1.3751898e-5, Iteration: 740
Current loss is: 1.3763961e-5, Iteration: 741
Current loss is: 1.3643136e-5, Iteration: 742
Current loss is: 1.3611027e-5, Iteration: 743

Current loss is: 1.3603656e-5, Iteration: 744
Current loss is: 1.3508657e-5, Iteration: 745
Current loss is: 1.3534247e-5, Iteration: 746
Current loss is: 1.3610568e-5, Iteration: 747
Current loss is: 1.33813155e-5, Iteration: 748
Current loss is: 1.33881895e-5, Iteration: 749
Current loss is: 1.3348618e-5, Iteration: 750
Current loss is: 1.34806105e-5, Iteration: 751
Current loss is: 1.3473738e-5, Iteration: 752
Current loss is: 1.3404259e-5, Iteration: 753
Current loss is: 1.3290721e-5, Iteration: 754
Current loss is: 1.3343687e-5, Iteration: 755
Current loss is: 1.3200775e-5, Iteration: 756
Current loss is: 1.3291766e-5, Iteration: 757
Current loss is: 1.32783325e-5, Iteration: 758
Current loss is: 1.3275305e-5, Iteration: 759
Current loss is: 1.3173438e-5, Iteration: 760
Current loss is: 1.3046071e-5, Iteration: 761
Current loss is: 1.3297969e-5, Iteration: 762
Current loss is: 1.2986387e-5, Iteration: 763
Current loss is: 1.3035093e-5, Iteration: 764
Current loss is: 1.3192118e-5, Iteration: 765
Current loss is: 1.3082721e-5, Iteration: 766
Current loss is: 1.2987706e-5, Iteration: 767
Current loss is: 1.30098915e-5, Iteration: 768
Current loss is: 1.2989424e-5, Iteration: 769
Current loss is: 1.2927785e-5, Iteration: 770
Current loss is: 1.2892345e-5, Iteration: 771
Current loss is: 1.2781589e-5, Iteration: 772
Current loss is: 1.2743896e-5, Iteration: 773
Current loss is: 1.2791153e-5, Iteration: 774
Current loss is: 1.2712462e-5, Iteration: 775
Current loss is: 1.2732728e-5, Iteration: 776
Current loss is: 1.2497381e-5, Iteration: 777
Current loss is: 1.2606084e-5, Iteration: 778
Current loss is: 1.2574842e-5, Iteration: 779
Current loss is: 1.2669384e-5, Iteration: 780
Current loss is: 1.2587874e-5, Iteration: 781
Current loss is: 1.2689681e-5, Iteration: 782
Current loss is: 1.2477325e-5, Iteration: 783
Current loss is: 1.245858e-5, Iteration: 784
Current loss is: 1.2606708e-5, Iteration: 785
Current loss is: 1.2517094e-5, Iteration: 786
Current loss is: 1.2435864e-5, Iteration: 787
Current loss is: 1.2456939e-5, Iteration: 788
Current loss is: 1.2392846e-5, Iteration: 789
Current loss is: 1.2514356e-5, Iteration: 790
Current loss is: 1.2320132e-5, Iteration: 791
Current loss is: 1.2295173e-5, Iteration: 792
Current loss is: 1.2173992e-5, Iteration: 793
Current loss is: 1.2269943e-5, Iteration: 794
Current loss is: 1.2289592e-5, Iteration: 795
Current loss is: 1.2315114e-5, Iteration: 796
Current loss is: 1.2335847e-5, Iteration: 797
Current loss is: 1.2220238e-5, Iteration: 798
Current loss is: 1.2262284e-5, Iteration: 799
Current loss is: 1.2106131e-5, Iteration: 800
Current loss is: 1.1959232e-5, Iteration: 801
Current loss is: 1.2205627e-5, Iteration: 802
Current loss is: 1.2005705e-5, Iteration: 803
Current loss is: 1.2171322e-5, Iteration: 804
Current loss is: 1.2052167e-5, Iteration: 805
Current loss is: 1.2091265e-5, Iteration: 806
Current loss is: 1.2044385e-5, Iteration: 807
Current loss is: 1.2028608e-5, Iteration: 808
Current loss is: 1.1965769e-5, Iteration: 809
Current loss is: 1.2006092e-5, Iteration: 810
Current loss is: 1.189905e-5, Iteration: 811
Current loss is: 1.1970922e-5, Iteration: 812
Current loss is: 1.1827593e-5, Iteration: 813
Current loss is: 1.18253865e-5, Iteration: 814
Current loss is: 1.1709797e-5, Iteration: 815
Current loss is: 1.1818696e-5, Iteration: 816
Current loss is: 1.173509e-5, Iteration: 817
Current loss is: 1.1758079e-5, Iteration: 818
Current loss is: 1.1772483e-5, Iteration: 819
Current loss is: 1.17417485e-5, Iteration: 820
Current loss is: 1.1772709e-5, Iteration: 821
Current loss is: 1.1735861e-5, Iteration: 822
Current loss is: 1.1659465e-5, Iteration: 823
Current loss is: 1.1583052e-5, Iteration: 824
Current loss is: 1.1515269e-5, Iteration: 825
Current loss is: 1.1618542e-5, Iteration: 826
Current loss is: 1.164929e-5, Iteration: 827
Current loss is: 1.1632043e-5, Iteration: 828
Current loss is: 1.1556439e-5, Iteration: 829
Current loss is: 1.1460697e-5, Iteration: 830
Current loss is: 1.1407201e-5, Iteration: 831
Current loss is: 1.1561617e-5, Iteration: 832
Current loss is: 1.1317252e-5, Iteration: 833
Current loss is: 1.1461265e-5, Iteration: 834
Current loss is: 1.1426555e-5, Iteration: 835
Current loss is: 1.1396709e-5, Iteration: 836
Current loss is: 1.1422296e-5, Iteration: 837
Current loss is: 1.1372121e-5, Iteration: 838
Current loss is: 1.1300017e-5, Iteration: 839
Current loss is: 1.1478966e-5, Iteration: 840
Current loss is: 1.1413507e-5, Iteration: 841
Current loss is: 1.1335491e-5, Iteration: 842
Current loss is: 1.1415855e-5, Iteration: 843
Current loss is: 1.13682545e-5, Iteration: 844
Current loss is: 1.1338237e-5, Iteration: 845
Current loss is: 1.1283135e-5, Iteration: 846
Current loss is: 1.12671205e-5, Iteration: 847
Current loss is: 1.1146173e-5, Iteration: 848
Current loss is: 1.11968875e-5, Iteration: 849
Current loss is: 1.1295109e-5, Iteration: 850
Current loss is: 1.1151604e-5, Iteration: 851
Current loss is: 1.1145107e-5, Iteration: 852
Current loss is: 1.0980488e-5, Iteration: 853
Current loss is: 1.0960098e-5, Iteration: 854
Current loss is: 1.1190399e-5, Iteration: 855
Current loss is: 1.1107762e-5, Iteration: 856
Current loss is: 1.1132235e-5, Iteration: 857
Current loss is: 1.1049623e-5, Iteration: 858
Current loss is: 1.1016993e-5, Iteration: 859
Current loss is: 1.10435285e-5, Iteration: 860
Current loss is: 1.1128005e-5, Iteration: 861
Current loss is: 1.0888819e-5, Iteration: 862
Current loss is: 1.0954262e-5, Iteration: 863
Current loss is: 1.0977703e-5, Iteration: 864
Current loss is: 1.09871235e-5, Iteration: 865
Current loss is: 1.0764028e-5, Iteration: 866
Current loss is: 1.0864952e-5, Iteration: 867
Current loss is: 1.0861097e-5, Iteration: 868
Current loss is: 1.0962103e-5, Iteration: 869
Current loss is: 1.0862537e-5, Iteration: 870
Current loss is: 1.0877102e-5, Iteration: 871
Current loss is: 1.0875145e-5, Iteration: 872
Current loss is: 1.0846397e-5, Iteration: 873
Current loss is: 1.0711803e-5, Iteration: 874
Current loss is: 1.0672562e-5, Iteration: 875
Current loss is: 1.0828352e-5, Iteration: 876
Current loss is: 1.073396e-5, Iteration: 877
Current loss is: 1.0678608e-5, Iteration: 878
Current loss is: 1.0813643e-5, Iteration: 879
Current loss is: 1.0610969e-5, Iteration: 880
Current loss is: 1.0675094e-5, Iteration: 881
Current loss is: 1.0639328e-5, Iteration: 882
Current loss is: 1.0693164e-5, Iteration: 883
Current loss is: 1.0668124e-5, Iteration: 884
Current loss is: 1.066249e-5, Iteration: 885
Current loss is: 1.0695059e-5, Iteration: 886
Current loss is: 1.0676829e-5, Iteration: 887
Current loss is: 1.0596591e-5, Iteration: 888
Current loss is: 1.073766e-5, Iteration: 889
Current loss is: 1.0510822e-5, Iteration: 890
Current loss is: 1.0673178e-5, Iteration: 891
Current loss is: 1.0524634e-5, Iteration: 892
Current loss is: 1.0484386e-5, Iteration: 893
Current loss is: 1.0415378e-5, Iteration: 894
Current loss is: 1.050026e-5, Iteration: 895
Current loss is: 1.0578651e-5, Iteration: 896
Current loss is: 1.0622685e-5, Iteration: 897
Current loss is: 1.0462509e-5, Iteration: 898
Current loss is: 1.0457555e-5, Iteration: 899
Current loss is: 1.0542351e-5, Iteration: 900
Current loss is: 1.0390483e-5, Iteration: 901
Current loss is: 1.0570599e-5, Iteration: 902
Current loss is: 1.0439779e-5, Iteration: 903
Current loss is: 1.03993025e-5, Iteration: 904
Current loss is: 1.0390165e-5, Iteration: 905
Current loss is: 1.0415022e-5, Iteration: 906
Current loss is: 1.0440646e-5, Iteration: 907
Current loss is: 1.03850325e-5, Iteration: 908
Current loss is: 1.042861e-5, Iteration: 909
Current loss is: 1.03609755e-5, Iteration: 910
Current loss is: 1.0238691e-5, Iteration: 911
Current loss is: 1.0286582e-5, Iteration: 912
Current loss is: 1.0363316e-5, Iteration: 913
Current loss is: 1.02648555e-5, Iteration: 914
Current loss is: 1.0248803e-5, Iteration: 915
Current loss is: 1.0350025e-5, Iteration: 916
Current loss is: 1.0271673e-5, Iteration: 917
Current loss is: 1.0397207e-5, Iteration: 918
Current loss is: 1.0154745e-5, Iteration: 919
Current loss is: 1.0227877e-5, Iteration: 920
Current loss is: 1.0230215e-5, Iteration: 921
Current loss is: 1.0338617e-5, Iteration: 922
Current loss is: 1.02775475e-5, Iteration: 923
Current loss is: 1.0176145e-5, Iteration: 924
Current loss is: 1.0099084e-5, Iteration: 925
Current loss is: 1.00784055e-5, Iteration: 926
Current loss is: 1.0120264e-5, Iteration: 927
Current loss is: 1.01654805e-5, Iteration: 928
Current loss is: 1.0024029e-5, Iteration: 929
Current loss is: 1.0134574e-5, Iteration: 930
Current loss is: 1.0030659e-5, Iteration: 931
Current loss is: 1.01075275e-5, Iteration: 932
Current loss is: 1.0133979e-5, Iteration: 933
Current loss is: 1.0112208e-5, Iteration: 934
Current loss is: 1.0274024e-5, Iteration: 935
Current loss is: 1.0040569e-5, Iteration: 936
Current loss is: 1.0149172e-5, Iteration: 937
Current loss is: 1.0077769e-5, Iteration: 938
Current loss is: 1.0040389e-5, Iteration: 939
Current loss is: 1.0271316e-5, Iteration: 940
Current loss is: 1.0040211e-5, Iteration: 941

Current loss is: 1.00307325e-5, Iteration: 942
Current loss is: 1.0037029e-5, Iteration: 943
Current loss is: 9.966901e-6, Iteration: 944
Current loss is: 1.0046761e-5, Iteration: 945
Current loss is: 9.909298e-6, Iteration: 946
Current loss is: 1.000852e-5, Iteration: 947
Current loss is: 9.907615e-6, Iteration: 948
Current loss is: 1.00534e-5, Iteration: 949
Current loss is: 9.9068e-6, Iteration: 950
Current loss is: 1.0011818e-5, Iteration: 951
Current loss is: 1.0015219e-5, Iteration: 952
Current loss is: 9.789224e-6, Iteration: 953
Current loss is: 1.00149155e-5, Iteration: 954
Current loss is: 9.890597e-6, Iteration: 955
Current loss is: 1.0001065e-5, Iteration: 956
Current loss is: 9.962841e-6, Iteration: 957
Current loss is: 9.879454e-6, Iteration: 958
Current loss is: 9.857771e-6, Iteration: 959
Current loss is: 9.97222e-6, Iteration: 960
Current loss is: 9.975731e-6, Iteration: 961
Current loss is: 9.831809e-6, Iteration: 962
Current loss is: 9.8302135e-6, Iteration: 963
Current loss is: 9.985634e-6, Iteration: 964
Current loss is: 9.89493e-6, Iteration: 965
Current loss is: 9.910243e-6, Iteration: 966
Current loss is: 9.809504e-6, Iteration: 967
Current loss is: 9.849018e-6, Iteration: 968
Current loss is: 9.785248e-6, Iteration: 969
Current loss is: 9.85883e-6, Iteration: 970
Current loss is: 9.695417e-6, Iteration: 971
Current loss is: 9.872953e-6, Iteration: 972
Current loss is: 9.804238e-6, Iteration: 973
Current loss is: 9.842552e-6, Iteration: 974
Current loss is: 9.92939e-6, Iteration: 975
Current loss is: 9.801662e-6, Iteration: 976
Current loss is: 9.770312e-6, Iteration: 977
Current loss is: 9.67476e-6, Iteration: 978
Current loss is: 9.738568e-6, Iteration: 979
Current loss is: 9.679461e-6, Iteration: 980
Current loss is: 9.683363e-6, Iteration: 981
Current loss is: 9.751509e-6, Iteration: 982
Current loss is: 9.783111e-6, Iteration: 983
Current loss is: 9.796164e-6, Iteration: 984
Current loss is: 9.695663e-6, Iteration: 985
Current loss is: 9.733141e-6, Iteration: 986
Current loss is: 9.698895e-6, Iteration: 987
Current loss is: 9.628966e-6, Iteration: 988
Current loss is: 9.719045e-6, Iteration: 989
Current loss is: 9.706214e-6, Iteration: 990
Current loss is: 9.700893e-6, Iteration: 991
Current loss is: 9.706064e-6, Iteration: 992
Current loss is: 9.689762e-6, Iteration: 993
Current loss is: 9.71013e-6, Iteration: 994
Current loss is: 9.535259e-6, Iteration: 995
Current loss is: 9.705946e-6, Iteration: 996
Current loss is: 9.6738295e-6, Iteration: 997
Current loss is: 9.5008145e-6, Iteration: 998
Current loss is: 9.60847e-6, Iteration: 999
Current loss is: 9.684645e-6, Iteration: 1000
Current loss is: 9.638682e-6, Iteration: 1001
Current loss is: 9.523033e-6, Iteration: 1002
Current loss is: 9.6796375e-6, Iteration: 1003
Current loss is: 9.685964e-6, Iteration: 1004
Current loss is: 9.575062e-6, Iteration: 1005
Current loss is: 9.593539e-6, Iteration: 1006
Current loss is: 9.586624e-6, Iteration: 1007
Current loss is: 9.5719115e-6, Iteration: 1008
Current loss is: 9.6432905e-6, Iteration: 1009
Current loss is: 9.584515e-6, Iteration: 1010
Current loss is: 9.681566e-6, Iteration: 1011
Current loss is: 9.457286e-6, Iteration: 1012
Current loss is: 9.61244e-6, Iteration: 1013
Current loss is: 9.526744e-6, Iteration: 1014
Current loss is: 9.638195e-6, Iteration: 1015
Current loss is: 9.647436e-6, Iteration: 1016
Current loss is: 9.45895e-6, Iteration: 1017
Current loss is: 9.54735e-6, Iteration: 1018
Current loss is: 9.46827e-6, Iteration: 1019
Current loss is: 9.538075e-6, Iteration: 1020
Current loss is: 9.540927e-6, Iteration: 1021
Current loss is: 9.463952e-6, Iteration: 1022
Current loss is: 9.5264795e-6, Iteration: 1023
Current loss is: 9.500629e-6, Iteration: 1024
Current loss is: 9.492289e-6, Iteration: 1025
Current loss is: 9.55632e-6, Iteration: 1026
Current loss is: 9.596888e-6, Iteration: 1027
Current loss is: 9.444071e-6, Iteration: 1028
Current loss is: 9.494997e-6, Iteration: 1029
Current loss is: 9.5873675e-6, Iteration: 1030
Current loss is: 9.4722e-6, Iteration: 1031
Current loss is: 9.584324e-6, Iteration: 1032
Current loss is: 9.425546e-6, Iteration: 1033
Current loss is: 9.396314e-6, Iteration: 1034
Current loss is: 9.357289e-6, Iteration: 1035
Current loss is: 9.450419e-6, Iteration: 1036
Current loss is: 9.469313e-6, Iteration: 1037
Current loss is: 9.402733e-6, Iteration: 1038
Current loss is: 9.46496e-6, Iteration: 1039
Current loss is: 9.275409e-6, Iteration: 1040
Current loss is: 9.511448e-6, Iteration: 1041
Current loss is: 9.427684e-6, Iteration: 1042
Current loss is: 9.332161e-6, Iteration: 1043
Current loss is: 9.493295e-6, Iteration: 1044
Current loss is: 9.388343e-6, Iteration: 1045
Current loss is: 9.4149655e-6, Iteration: 1046
Current loss is: 9.45773e-6, Iteration: 1047
Current loss is: 9.399026e-6, Iteration: 1048
Current loss is: 9.442021e-6, Iteration: 1049
Current loss is: 9.432073e-6, Iteration: 1050
Current loss is: 9.389116e-6, Iteration: 1051
Current loss is: 9.375821e-6, Iteration: 1052
Current loss is: 9.497224e-6, Iteration: 1053
Current loss is: 9.307686e-6, Iteration: 1054
Current loss is: 9.283278e-6, Iteration: 1055
Current loss is: 9.439525e-6, Iteration: 1056
Current loss is: 9.412427e-6, Iteration: 1057
Current loss is: 9.3603985e-6, Iteration: 1058
Current loss is: 9.432793e-6, Iteration: 1059
Current loss is: 9.403627e-6, Iteration: 1060
Current loss is: 9.280235e-6, Iteration: 1061
Current loss is: 9.595829e-6, Iteration: 1062
Current loss is: 9.272922e-6, Iteration: 1063
Current loss is: 9.333942e-6, Iteration: 1064
Current loss is: 9.27582e-6, Iteration: 1065
Current loss is: 9.347147e-6, Iteration: 1066
Current loss is: 9.397703e-6, Iteration: 1067
Current loss is: 9.391051e-6, Iteration: 1068
Current loss is: 9.4435745e-6, Iteration: 1069
Current loss is: 9.35392e-6, Iteration: 1070
Current loss is: 9.386047e-6, Iteration: 1071
Current loss is: 9.338368e-6, Iteration: 1072
Current loss is: 9.381757e-6, Iteration: 1073
Current loss is: 9.314393e-6, Iteration: 1074
Current loss is: 9.206482e-6, Iteration: 1075
Current loss is: 9.210345e-6, Iteration: 1076
Current loss is: 9.288063e-6, Iteration: 1077
Current loss is: 9.245194e-6, Iteration: 1078
Current loss is: 9.2575665e-6, Iteration: 1079
Current loss is: 9.294676e-6, Iteration: 1080
Current loss is: 9.3426725e-6, Iteration: 1081
Current loss is: 9.285477e-6, Iteration: 1082
Current loss is: 9.294774e-6, Iteration: 1083
Current loss is: 9.259112e-6, Iteration: 1084
Current loss is: 9.283496e-6, Iteration: 1085
Current loss is: 9.356795e-6, Iteration: 1086
Current loss is: 9.245614e-6, Iteration: 1087
Current loss is: 9.249777e-6, Iteration: 1088
Current loss is: 9.154354e-6, Iteration: 1089
Current loss is: 9.238609e-6, Iteration: 1090
Current loss is: 9.123355e-6, Iteration: 1091
Current loss is: 9.352138e-6, Iteration: 1092
Current loss is: 9.218971e-6, Iteration: 1093
Current loss is: 9.170304e-6, Iteration: 1094
Current loss is: 9.250523e-6, Iteration: 1095
Current loss is: 9.1816655e-6, Iteration: 1096
Current loss is: 9.257684e-6, Iteration: 1097
Current loss is: 9.346038e-6, Iteration: 1098
Current loss is: 9.220248e-6, Iteration: 1099
Current loss is: 9.213298e-6, Iteration: 1100
Current loss is: 9.324964e-6, Iteration: 1101
Current loss is: 9.191985e-6, Iteration: 1102
Current loss is: 9.202825e-6, Iteration: 1103
Current loss is: 9.304424e-6, Iteration: 1104
Current loss is: 9.217883e-6, Iteration: 1105
Current loss is: 9.366825e-6, Iteration: 1106
Current loss is: 9.19917e-6, Iteration: 1107
Current loss is: 9.316758e-6, Iteration: 1108
Current loss is: 9.234815e-6, Iteration: 1109
Current loss is: 9.118569e-6, Iteration: 1110
Current loss is: 9.233727e-6, Iteration: 1111
Current loss is: 9.211989e-6, Iteration: 1112
Current loss is: 9.2900755e-6, Iteration: 1113
Current loss is: 9.220991e-6, Iteration: 1114
Current loss is: 9.2954415e-6, Iteration: 1115
Current loss is: 9.128904e-6, Iteration: 1116
Current loss is: 9.290584e-6, Iteration: 1117
Current loss is: 9.238358e-6, Iteration: 1118
Current loss is: 9.162367e-6, Iteration: 1119
Current loss is: 9.193778e-6, Iteration: 1120
Current loss is: 9.304504e-6, Iteration: 1121
Current loss is: 9.1823385e-6, Iteration: 1122
Current loss is: 9.220775e-6, Iteration: 1123
Current loss is: 9.154138e-6, Iteration: 1124
Current loss is: 9.178174e-6, Iteration: 1125
Current loss is: 9.130158e-6, Iteration: 1126
Current loss is: 9.253563e-6, Iteration: 1127
Current loss is: 9.18928e-6, Iteration: 1128
Current loss is: 9.2622395e-6, Iteration: 1129
Current loss is: 9.189676e-6, Iteration: 1130
Current loss is: 9.236562e-6, Iteration: 1131
Current loss is: 9.142782e-6, Iteration: 1132
Current loss is: 9.384137e-6, Iteration: 1133
Current loss is: 9.196276e-6, Iteration: 1134
Current loss is: 9.064392e-6, Iteration: 1135

Current loss is: 9.1153315e-6, Iteration: 1136
Current loss is: 9.159743e-6, Iteration: 1137
Current loss is: 9.121164e-6, Iteration: 1138
Current loss is: 9.125717e-6, Iteration: 1139
Current loss is: 9.204867e-6, Iteration: 1140
Current loss is: 9.0782505e-6, Iteration: 1141
Current loss is: 8.996843e-6, Iteration: 1142
Current loss is: 9.210213e-6, Iteration: 1143
Current loss is: 9.180901e-6, Iteration: 1144
Current loss is: 9.1095535e-6, Iteration: 1145
Current loss is: 9.232588e-6, Iteration: 1146
Current loss is: 9.118248e-6, Iteration: 1147
Current loss is: 9.20242e-6, Iteration: 1148
Current loss is: 9.213451e-6, Iteration: 1149
Current loss is: 9.148311e-6, Iteration: 1150
Current loss is: 9.100383e-6, Iteration: 1151
Current loss is: 9.198775e-6, Iteration: 1152
Current loss is: 9.141033e-6, Iteration: 1153
Current loss is: 9.163207e-6, Iteration: 1154
Current loss is: 9.218221e-6, Iteration: 1155
Current loss is: 9.109894e-6, Iteration: 1156
Current loss is: 9.036265e-6, Iteration: 1157
Current loss is: 9.093072e-6, Iteration: 1158
Current loss is: 8.929785e-6, Iteration: 1159
Current loss is: 9.110585e-6, Iteration: 1160
Current loss is: 9.061271e-6, Iteration: 1161
Current loss is: 9.151572e-6, Iteration: 1162
Current loss is: 9.15165e-6, Iteration: 1163
Current loss is: 9.16752e-6, Iteration: 1164
Current loss is: 8.99713e-6, Iteration: 1165
Current loss is: 9.059501e-6, Iteration: 1166
Current loss is: 9.100511e-6, Iteration: 1167
Current loss is: 9.1078455e-6, Iteration: 1168
Current loss is: 8.96147e-6, Iteration: 1169
Current loss is: 9.097326e-6, Iteration: 1170
Current loss is: 9.070658e-6, Iteration: 1171
Current loss is: 9.038531e-6, Iteration: 1172
Current loss is: 9.097715e-6, Iteration: 1173
Current loss is: 9.256615e-6, Iteration: 1174
Current loss is: 9.134224e-6, Iteration: 1175
Current loss is: 9.080941e-6, Iteration: 1176
Current loss is: 9.070241e-6, Iteration: 1177
Current loss is: 9.059605e-6, Iteration: 1178
Current loss is: 9.1196525e-6, Iteration: 1179
Current loss is: 9.108699e-6, Iteration: 1180
Current loss is: 9.074016e-6, Iteration: 1181
Current loss is: 8.933529e-6, Iteration: 1182
Current loss is: 9.068515e-6, Iteration: 1183
Current loss is: 8.934853e-6, Iteration: 1184
Current loss is: 9.173269e-6, Iteration: 1185
Current loss is: 8.966147e-6, Iteration: 1186
Current loss is: 9.0856665e-6, Iteration: 1187
Current loss is: 9.084567e-6, Iteration: 1188
Current loss is: 8.990963e-6, Iteration: 1189
Current loss is: 8.972788e-6, Iteration: 1190
Current loss is: 8.904801e-6, Iteration: 1191
Current loss is: 9.107023e-6, Iteration: 1192
Current loss is: 9.052446e-6, Iteration: 1193
Current loss is: 8.949955e-6, Iteration: 1194
Current loss is: 9.007313e-6, Iteration: 1195
Current loss is: 8.916491e-6, Iteration: 1196
Current loss is: 9.136168e-6, Iteration: 1197
Current loss is: 8.9690875e-6, Iteration: 1198
Current loss is: 9.101159e-6, Iteration: 1199
Current loss is: 8.997172e-6, Iteration: 1200
Current loss is: 9.055708e-6, Iteration: 1201
Current loss is: 8.939371e-6, Iteration: 1202
Current loss is: 9.016891e-6, Iteration: 1203
Current loss is: 9.070673e-6, Iteration: 1204
Current loss is: 9.0168e-6, Iteration: 1205
Current loss is: 9.005598e-6, Iteration: 1206
Current loss is: 8.968396e-6, Iteration: 1207
Current loss is: 8.871359e-6, Iteration: 1208
Current loss is: 8.938339e-6, Iteration: 1209
Current loss is: 9.0561325e-6, Iteration: 1210
Current loss is: 8.931224e-6, Iteration: 1211
Current loss is: 9.171396e-6, Iteration: 1212
Current loss is: 9.059859e-6, Iteration: 1213
Current loss is: 9.017178e-6, Iteration: 1214
Current loss is: 8.988325e-6, Iteration: 1215
Current loss is: 9.100192e-6, Iteration: 1216
Current loss is: 9.109161e-6, Iteration: 1217
Current loss is: 9.033358e-6, Iteration: 1218
Current loss is: 9.065952e-6, Iteration: 1219
Current loss is: 8.895219e-6, Iteration: 1220
Current loss is: 9.011401e-6, Iteration: 1221
Current loss is: 9.141605e-6, Iteration: 1222
Current loss is: 9.065025e-6, Iteration: 1223
Current loss is: 9.030212e-6, Iteration: 1224
Current loss is: 8.979185e-6, Iteration: 1225
Current loss is: 9.0690755e-6, Iteration: 1226
Current loss is: 8.99299e-6, Iteration: 1227
Current loss is: 9.044323e-6, Iteration: 1228
Current loss is: 9.026244e-6, Iteration: 1229
Current loss is: 9.051135e-6, Iteration: 1230
Current loss is: 9.002622e-6, Iteration: 1231
Current loss is: 9.082303e-6, Iteration: 1232
Current loss is: 9.096651e-6, Iteration: 1233
Current loss is: 8.956714e-6, Iteration: 1234
Current loss is: 9.0341755e-6, Iteration: 1235
Current loss is: 8.931702e-6, Iteration: 1236
Current loss is: 8.945128e-6, Iteration: 1237
Current loss is: 9.511105e-6, Iteration: 1238
Current loss is: 9.020409e-6, Iteration: 1239
Current loss is: 9.0829035e-6, Iteration: 1240
Current loss is: 8.973873e-6, Iteration: 1241
Current loss is: 8.9551895e-6, Iteration: 1242
Current loss is: 8.908141e-6, Iteration: 1243
Current loss is: 9.06951e-6, Iteration: 1244
Current loss is: 8.950717e-6, Iteration: 1245
Current loss is: 9.4727375e-6, Iteration: 1246
Current loss is: 8.932041e-6, Iteration: 1247
Current loss is: 9.000218e-6, Iteration: 1248
Current loss is: 8.963298e-6, Iteration: 1249
Current loss is: 8.93989e-6, Iteration: 1250
Current loss is: 8.9260675e-6, Iteration: 1251
Current loss is: 8.983904e-6, Iteration: 1252
Current loss is: 8.942453e-6, Iteration: 1253
Current loss is: 8.931967e-6, Iteration: 1254
Current loss is: 9.436282e-6, Iteration: 1255
Current loss is: 9.011929e-6, Iteration: 1256
Current loss is: 8.946985e-6, Iteration: 1257
Current loss is: 9.000379e-6, Iteration: 1258
Current loss is: 8.950714e-6, Iteration: 1259
Current loss is: 9.502329e-6, Iteration: 1260
Current loss is: 8.992241e-6, Iteration: 1261
Current loss is: 8.945157e-6, Iteration: 1262
Current loss is: 8.902767e-6, Iteration: 1263
Current loss is: 8.921297e-6, Iteration: 1264
Current loss is: 8.919818e-6, Iteration: 1265
Current loss is: 8.949271e-6, Iteration: 1266
Current loss is: 8.993943e-6, Iteration: 1267
Current loss is: 8.931354e-6, Iteration: 1268
Current loss is: 9.010916e-6, Iteration: 1269
Current loss is: 9.438843e-6, Iteration: 1270
Current loss is: 9.472791e-6, Iteration: 1271
Current loss is: 9.542963e-6, Iteration: 1272
Current loss is: 9.017358e-6, Iteration: 1273
Current loss is: 9.479354e-6, Iteration: 1274
Current loss is: 9.3951885e-6, Iteration: 1275
Current loss is: 8.935806e-6, Iteration: 1276
Current loss is: 9.51495e-6, Iteration: 1277
Current loss is: 8.949866e-6, Iteration: 1278
Current loss is: 8.931961e-6, Iteration: 1279
Current loss is: 8.94587e-6, Iteration: 1280
Current loss is: 8.834e-6, Iteration: 1281
Current loss is: 8.929426e-6, Iteration: 1282
Current loss is: 8.91502e-6, Iteration: 1283
Current loss is: 8.947297e-6, Iteration: 1284
Current loss is: 9.47506e-6, Iteration: 1285
Current loss is: 8.983787e-6, Iteration: 1286
Current loss is: 8.894857e-6, Iteration: 1287
Current loss is: 9.538668e-6, Iteration: 1288
Current loss is: 9.481003e-6, Iteration: 1289
Current loss is: 9.013997e-6, Iteration: 1290
Current loss is: 8.890421e-6, Iteration: 1291
Current loss is: 8.963129e-6, Iteration: 1292
Current loss is: 9.54956e-6, Iteration: 1293

Current loss is: 8.839484e-6, Iteration: 1294
Current loss is: 8.942595e-6, Iteration: 1295
Current loss is: 8.873541e-6, Iteration: 1296
Current loss is: 9.480478e-6, Iteration: 1297
Current loss is: 9.4969855e-6, Iteration: 1298
Current loss is: 8.972228e-6, Iteration: 1299
Current loss is: 8.981157e-6, Iteration: 1300
Current loss is: 8.841526e-6, Iteration: 1301
Current loss is: 8.92229e-6, Iteration: 1302
Current loss is: 8.92762e-6, Iteration: 1303
Current loss is: 8.834442e-6, Iteration: 1304
Current loss is: 8.89298e-6, Iteration: 1305
Current loss is: 8.891507e-6, Iteration: 1306
Current loss is: 9.495288e-6, Iteration: 1307
Current loss is: 8.912071e-6, Iteration: 1308
Current loss is: 8.8551515e-6, Iteration: 1309
Current loss is: 9.446081e-6, Iteration: 1310
Current loss is: 8.867325e-6, Iteration: 1311
Current loss is: 9.389257e-6, Iteration: 1312
Current loss is: 9.431468e-6, Iteration: 1313
Current loss is: 9.013111e-6, Iteration: 1314
Current loss is: 8.914762e-6, Iteration: 1315
Current loss is: 8.93227e-6, Iteration: 1316
Current loss is: 8.842051e-6, Iteration: 1317
Current loss is: 8.887339e-6, Iteration: 1318
Current loss is: 8.907085e-6, Iteration: 1319
Current loss is: 8.803568e-6, Iteration: 1320
Current loss is: 8.928285e-6, Iteration: 1321
Current loss is: 9.3782455e-6, Iteration: 1322
Current loss is: 8.905017e-6, Iteration: 1323
Current loss is: 8.94545e-6, Iteration: 1324
Current loss is: 9.436509e-6, Iteration: 1325
Current loss is: 9.395633e-6, Iteration: 1326
Current loss is: 9.479432e-6, Iteration: 1327
Current loss is: 8.9289315e-6, Iteration: 1328
Current loss is: 9.550338e-6, Iteration: 1329
Current loss is: 8.932508e-6, Iteration: 1330
Current loss is: 9.461337e-6, Iteration: 1331
Current loss is: 8.884284e-6, Iteration: 1332
Current loss is: 8.8526e-6, Iteration: 1333
Current loss is: 8.820629e-6, Iteration: 1334
Current loss is: 8.871504e-6, Iteration: 1335
Current loss is: 9.405816e-6, Iteration: 1336
Current loss is: 8.932975e-6, Iteration: 1337
Current loss is: 9.40207e-6, Iteration: 1338
Current loss is: 9.403404e-6, Iteration: 1339
Current loss is: 9.51895e-6, Iteration: 1340
Current loss is: 9.329417e-6, Iteration: 1341
Current loss is: 8.779515e-6, Iteration: 1342
Current loss is: 9.382471e-6, Iteration: 1343
Current loss is: 8.857024e-6, Iteration: 1344
Current loss is: 9.325132e-6, Iteration: 1345
Current loss is: 9.4781435e-6, Iteration: 1346
Current loss is: 8.9363075e-6, Iteration: 1347
Current loss is: 9.508497e-6, Iteration: 1348
Current loss is: 9.381033e-6, Iteration: 1349
Current loss is: 8.737858e-6, Iteration: 1350
Current loss is: 9.3832205e-6, Iteration: 1351
Current loss is: 9.286667e-6, Iteration: 1352
Current loss is: 8.855448e-6, Iteration: 1353
Current loss is: 9.358846e-6, Iteration: 1354
Current loss is: 9.390451e-6, Iteration: 1355
Current loss is: 9.292566e-6, Iteration: 1356
Current loss is: 8.794794e-6, Iteration: 1357
Current loss is: 8.852034e-6, Iteration: 1358
Current loss is: 9.325519e-6, Iteration: 1359
Current loss is: 9.378882e-6, Iteration: 1360
Current loss is: 8.926864e-6, Iteration: 1361
Current loss is: 9.422267e-6, Iteration: 1362
Current loss is: 9.363886e-6, Iteration: 1363
Current loss is: 8.799566e-6, Iteration: 1364
Current loss is: 8.796287e-6, Iteration: 1365
Current loss is: 9.469026e-6, Iteration: 1366
Current loss is: 8.828788e-6, Iteration: 1367
Current loss is: 9.375399e-6, Iteration: 1368
Current loss is: 8.937803e-6, Iteration: 1369
Current loss is: 9.386744e-6, Iteration: 1370
Current loss is: 9.406691e-6, Iteration: 1371
Current loss is: 8.83931e-6, Iteration: 1372
Current loss is: 9.4018405e-6, Iteration: 1373
Current loss is: 9.2913115e-6, Iteration: 1374
Current loss is: 9.477675e-6, Iteration: 1375
Current loss is: 9.432545e-6, Iteration: 1376
Current loss is: 8.94257e-6, Iteration: 1377
Current loss is: 9.341787e-6, Iteration: 1378
Current loss is: 8.8961115e-6, Iteration: 1379
Current loss is: 9.4180705e-6, Iteration: 1380
Current loss is: 9.337091e-6, Iteration: 1381
Current loss is: 8.927697e-6, Iteration: 1382
Current loss is: 8.80908e-6, Iteration: 1383
Current loss is: 8.851197e-6, Iteration: 1384
Current loss is: 9.320209e-6, Iteration: 1385
Current loss is: 8.953858e-6, Iteration: 1386
Current loss is: 9.32645e-6, Iteration: 1387
Current loss is: 9.3994e-6, Iteration: 1388
Current loss is: 8.879282e-6, Iteration: 1389
Current loss is: 9.299412e-6, Iteration: 1390
Current loss is: 9.449663e-6, Iteration: 1391
Current loss is: 8.940961e-6, Iteration: 1392
Current loss is: 9.439192e-6, Iteration: 1393
Current loss is: 9.2995315e-6, Iteration: 1394
Current loss is: 8.911549e-6, Iteration: 1395

Current loss is: 8.785646e-6, Iteration: 1396
Current loss is: 8.826721e-6, Iteration: 1397
Current loss is: 9.395275e-6, Iteration: 1398
Current loss is: 9.330281e-6, Iteration: 1399
Current loss is: 8.834835e-6, Iteration: 1400
Current loss is: 8.864757e-6, Iteration: 1401
Current loss is: 9.31463e-6, Iteration: 1402
Current loss is: 9.281002e-6, Iteration: 1403
Current loss is: 9.395357e-6, Iteration: 1404
Current loss is: 9.315672e-6, Iteration: 1405
Current loss is: 9.272431e-6, Iteration: 1406
Current loss is: 9.386165e-6, Iteration: 1407
Current loss is: 8.776735e-6, Iteration: 1408
Current loss is: 8.805948e-6, Iteration: 1409
Current loss is: 9.298046e-6, Iteration: 1410
Current loss is: 8.885373e-6, Iteration: 1411
Current loss is: 9.281809e-6, Iteration: 1412
Current loss is: 9.2730725e-6, Iteration: 1413
Current loss is: 9.32686e-6, Iteration: 1414
Current loss is: 9.353234e-6, Iteration: 1415
Current loss is: 8.856192e-6, Iteration: 1416
Current loss is: 8.678266e-6, Iteration: 1417
Current loss is: 8.730389e-6, Iteration: 1418
Current loss is: 9.354949e-6, Iteration: 1419
Current loss is: 9.303136e-6, Iteration: 1420
Current loss is: 8.778732e-6, Iteration: 1421
Current loss is: 9.249124e-6, Iteration: 1422
Current loss is: 8.76053e-6, Iteration: 1423
Current loss is: 9.250962e-6, Iteration: 1424
Current loss is: 8.797532e-6, Iteration: 1425
Current loss is: 8.643753e-6, Iteration: 1426
Current loss is: 8.772157e-6, Iteration: 1427
Current loss is: 9.182008e-6, Iteration: 1428
Current loss is: 9.257743e-6, Iteration: 1429
Current loss is: 9.371389e-6, Iteration: 1430
Current loss is: 9.265417e-6, Iteration: 1431
Current loss is: 9.373114e-6, Iteration: 1432
Current loss is: 9.224847e-6, Iteration: 1433
Current loss is: 8.8114775e-6, Iteration: 1434
Current loss is: 8.756519e-6, Iteration: 1435
Current loss is: 9.365924e-6, Iteration: 1436
Current loss is: 9.3058225e-6, Iteration: 1437
Current loss is: 9.257093e-6, Iteration: 1438
Current loss is: 9.264231e-6, Iteration: 1439
Current loss is: 8.847706e-6, Iteration: 1440
Current loss is: 9.311794e-6, Iteration: 1441
Current loss is: 9.31144e-6, Iteration: 1442
Current loss is: 9.263971e-6, Iteration: 1443
Current loss is: 8.7583685e-6, Iteration: 1444
Current loss is: 8.694298e-6, Iteration: 1445
Current loss is: 9.278987e-6, Iteration: 1446
Current loss is: 9.26094e-6, Iteration: 1447
Current loss is: 9.2652035e-6, Iteration: 1448
Current loss is: 9.357587e-6, Iteration: 1449
Current loss is: 9.274919e-6, Iteration: 1450
Current loss is: 9.282894e-6, Iteration: 1451
Current loss is: 9.32577e-6, Iteration: 1452
Current loss is: 9.296505e-6, Iteration: 1453
Current loss is: 8.818041e-6, Iteration: 1454
Current loss is: 8.751344e-6, Iteration: 1455
Current loss is: 9.335912e-6, Iteration: 1456
Current loss is: 9.354577e-6, Iteration: 1457
Current loss is: 9.308281e-6, Iteration: 1458
Current loss is: 9.252418e-6, Iteration: 1459
Current loss is: 9.289432e-6, Iteration: 1460
Current loss is: 8.5754e-6, Iteration: 1461
Current loss is: 8.789968e-6, Iteration: 1462
Current loss is: 9.256317e-6, Iteration: 1463
Current loss is: 9.271866e-6, Iteration: 1464
Current loss is: 8.864066e-6, Iteration: 1465
Current loss is: 8.738042e-6, Iteration: 1466
Current loss is: 8.700896e-6, Iteration: 1467
Current loss is: 9.278314e-6, Iteration: 1468
Current loss is: 8.667699e-6, Iteration: 1469
Current loss is: 8.834371e-6, Iteration: 1470
Current loss is: 8.7548615e-6, Iteration: 1471
Current loss is: 9.2726095e-6, Iteration: 1472
Current loss is: 9.341515e-6, Iteration: 1473
Current loss is: 9.195574e-6, Iteration: 1474
Current loss is: 9.335048e-6, Iteration: 1475
Current loss is: 8.719931e-6, Iteration: 1476
Current loss is: 9.325589e-6, Iteration: 1477
Current loss is: 8.650091e-6, Iteration: 1478
Current loss is: 8.707356e-6, Iteration: 1479
Current loss is: 8.662886e-6, Iteration: 1480
Current loss is: 9.299818e-6, Iteration: 1481
Current loss is: 9.336791e-6, Iteration: 1482
Current loss is: 9.311251e-6, Iteration: 1483
Current loss is: 9.229465e-6, Iteration: 1484

Current loss is: 8.764431e-6, Iteration: 1485
Current loss is: 9.279701e-6, Iteration: 1486
Current loss is: 9.173927e-6, Iteration: 1487
Current loss is: 9.316793e-6, Iteration: 1488
Current loss is: 9.323741e-6, Iteration: 1489
Current loss is: 9.2213795e-6, Iteration: 1490
Current loss is: 9.293371e-6, Iteration: 1491
Current loss is: 9.215704e-6, Iteration: 1492
Current loss is: 9.182919e-6, Iteration: 1493
Current loss is: 9.225124e-6, Iteration: 1494
Current loss is: 9.235138e-6, Iteration: 1495
Current loss is: 9.115611e-6, Iteration: 1496
Current loss is: 8.595398e-6, Iteration: 1497
Current loss is: 8.681508e-6, Iteration: 1498
Current loss is: 9.282999e-6, Iteration: 1499
Current loss is: 9.223666e-6, Iteration: 1500
Current loss is: 9.161546e-6, Iteration: 1501
Current loss is: 9.284977e-6, Iteration: 1502
Current loss is: 9.150275e-6, Iteration: 1503
Current loss is: 9.266701e-6, Iteration: 1504
Current loss is: 9.3101635e-6, Iteration: 1505
Current loss is: 8.705945e-6, Iteration: 1506
Current loss is: 9.068361e-6, Iteration: 1507
Current loss is: 9.251661e-6, Iteration: 1508
Current loss is: 8.666917e-6, Iteration: 1509
Current loss is: 9.221778e-6, Iteration: 1510
Current loss is: 8.668577e-6, Iteration: 1511
Current loss is: 8.7030685e-6, Iteration: 1512
Current loss is: 9.260519e-6, Iteration: 1513
Current loss is: 9.204878e-6, Iteration: 1514
Current loss is: 9.209838e-6, Iteration: 1515
Current loss is: 9.2797e-6, Iteration: 1516
Current loss is: 9.185788e-6, Iteration: 1517
Current loss is: 9.15348e-6, Iteration: 1518
Current loss is: 9.119939e-6, Iteration: 1519
Current loss is: 9.196076e-6, Iteration: 1520
Current loss is: 9.225885e-6, Iteration: 1521
Current loss is: 9.122592e-6, Iteration: 1522
Current loss is: 8.749248e-6, Iteration: 1523
Current loss is: 8.746627e-6, Iteration: 1524
Current loss is: 9.1363045e-6, Iteration: 1525
Current loss is: 8.725205e-6, Iteration: 1526
Current loss is: 8.748313e-6, Iteration: 1527
Current loss is: 8.6965865e-6, Iteration: 1528
Current loss is: 9.188006e-6, Iteration: 1529
Current loss is: 9.179119e-6, Iteration: 1530
Current loss is: 9.211897e-6, Iteration: 1531
Current loss is: 8.671027e-6, Iteration: 1532
Current loss is: 9.147484e-6, Iteration: 1533
Current loss is: 9.15339e-6, Iteration: 1534
Current loss is: 8.703093e-6, Iteration: 1535
Current loss is: 8.634195e-6, Iteration: 1536
Current loss is: 9.202499e-6, Iteration: 1537
Current loss is: 9.169757e-6, Iteration: 1538
Current loss is: 8.574937e-6, Iteration: 1539
Current loss is: 9.2773535e-6, Iteration: 1540
Current loss is: 8.748872e-6, Iteration: 1541
Current loss is: 9.193887e-6, Iteration: 1542
Current loss is: 9.152364e-6, Iteration: 1543
Current loss is: 9.143678e-6, Iteration: 1544
Current loss is: 9.200267e-6, Iteration: 1545
Current loss is: 9.137273e-6, Iteration: 1546
Current loss is: 9.202206e-6, Iteration: 1547
Current loss is: 9.149302e-6, Iteration: 1548
Current loss is: 9.238115e-6, Iteration: 1549
Current loss is: 9.102224e-6, Iteration: 1550
Current loss is: 8.646198e-6, Iteration: 1551
Current loss is: 8.681778e-6, Iteration: 1552
Current loss is: 9.224396e-6, Iteration: 1553
Current loss is: 9.179277e-6, Iteration: 1554
Current loss is: 9.153725e-6, Iteration: 1555
Current loss is: 8.680403e-6, Iteration: 1556
Current loss is: 9.0998055e-6, Iteration: 1557
Current loss is: 9.148596e-6, Iteration: 1558
Current loss is: 9.145451e-6, Iteration: 1559
Current loss is: 9.117871e-6, Iteration: 1560
Current loss is: 9.184765e-6, Iteration: 1561
Current loss is: 9.129152e-6, Iteration: 1562
Current loss is: 9.19567e-6, Iteration: 1563
Current loss is: 9.098623e-6, Iteration: 1564
Current loss is: 9.196931e-6, Iteration: 1565
Current loss is: 9.170212e-6, Iteration: 1566
Current loss is: 9.054705e-6, Iteration: 1567
Current loss is: 8.99318e-6, Iteration: 1568
Current loss is: 9.130847e-6, Iteration: 1569
Current loss is: 9.159934e-6, Iteration: 1570
Current loss is: 9.148958e-6, Iteration: 1571
Current loss is: 8.66193e-6, Iteration: 1572
Current loss is: 9.200796e-6, Iteration: 1573

Current loss is: 9.151474e-6, Iteration: 1574
Current loss is: 9.0545345e-6, Iteration: 1575
Current loss is: 9.096773e-6, Iteration: 1576
Current loss is: 9.233151e-6, Iteration: 1577
Current loss is: 9.16012e-6, Iteration: 1578
Current loss is: 9.175139e-6, Iteration: 1579
Current loss is: 8.621941e-6, Iteration: 1580
Current loss is: 9.172725e-6, Iteration: 1581
Current loss is: 9.15007e-6, Iteration: 1582
Current loss is: 8.663398e-6, Iteration: 1583
Current loss is: 9.12501e-6, Iteration: 1584
Current loss is: 9.128692e-6, Iteration: 1585
Current loss is: 9.089527e-6, Iteration: 1586
Current loss is: 9.093305e-6, Iteration: 1587
Current loss is: 9.1412e-6, Iteration: 1588
Current loss is: 8.463187e-6, Iteration: 1589
Current loss is: 9.112367e-6, Iteration: 1590
Current loss is: 8.562852e-6, Iteration: 1591
Current loss is: 9.116553e-6, Iteration: 1592
Current loss is: 9.179024e-6, Iteration: 1593
Current loss is: 9.025584e-6, Iteration: 1594
Current loss is: 9.119273e-6, Iteration: 1595
Current loss is: 8.6164555e-6, Iteration: 1596
Current loss is: 9.123521e-6, Iteration: 1597
Current loss is: 9.159934e-6, Iteration: 1598
Current loss is: 8.4990315e-6, Iteration: 1599
Current loss is: 9.042926e-6, Iteration: 1600
Current loss is: 9.182095e-6, Iteration: 1601
Current loss is: 9.107093e-6, Iteration: 1602
Current loss is: 8.73012e-6, Iteration: 1603
Current loss is: 9.108487e-6, Iteration: 1604
Current loss is: 9.19866e-6, Iteration: 1605
Current loss is: 8.998022e-6, Iteration: 1606
Current loss is: 9.166824e-6, Iteration: 1607
Current loss is: 9.17955e-6, Iteration: 1608
Current loss is: 8.661994e-6, Iteration: 1609
Current loss is: 8.581941e-6, Iteration: 1610
Current loss is: 9.08786e-6, Iteration: 1611
Current loss is: 9.083529e-6, Iteration: 1612
Current loss is: 8.994561e-6, Iteration: 1613
Current loss is: 9.01592e-6, Iteration: 1614
Current loss is: 9.125317e-6, Iteration: 1615
Current loss is: 9.062608e-6, Iteration: 1616
Current loss is: 9.062872e-6, Iteration: 1617
Current loss is: 9.121653e-6, Iteration: 1618
Current loss is: 9.0888425e-6, Iteration: 1619
Current loss is: 9.059669e-6, Iteration: 1620
Current loss is: 8.509121e-6, Iteration: 1621
Current loss is: 9.02599e-6, Iteration: 1622
Current loss is: 9.116689e-6, Iteration: 1623
Current loss is: 8.446125e-6, Iteration: 1624
Current loss is: 9.147783e-6, Iteration: 1625
Current loss is: 9.104639e-6, Iteration: 1626
Current loss is: 8.517866e-6, Iteration: 1627
Current loss is: 8.648836e-6, Iteration: 1628
Current loss is: 8.622955e-6, Iteration: 1629
Current loss is: 9.004595e-6, Iteration: 1630
Current loss is: 9.135463e-6, Iteration: 1631
Current loss is: 9.065604e-6, Iteration: 1632
Current loss is: 8.513456e-6, Iteration: 1633
Current loss is: 8.572841e-6, Iteration: 1634
Current loss is: 8.6248965e-6, Iteration: 1635
Current loss is: 8.442523e-6, Iteration: 1636
Current loss is: 8.578684e-6, Iteration: 1637
Current loss is: 9.045523e-6, Iteration: 1638
Current loss is: 8.555047e-6, Iteration: 1639
Current loss is: 8.597531e-6, Iteration: 1640
Current loss is: 9.034967e-6, Iteration: 1641
Current loss is: 9.077431e-6, Iteration: 1642
Current loss is: 8.51663e-6, Iteration: 1643
Current loss is: 8.597936e-6, Iteration: 1644
Current loss is: 8.438823e-6, Iteration: 1645
Current loss is: 9.00622e-6, Iteration: 1646
Current loss is: 8.5718e-6, Iteration: 1647
Current loss is: 8.9969135e-6, Iteration: 1648
Current loss is: 8.5195525e-6, Iteration: 1649
Current loss is: 8.535544e-6, Iteration: 1650
Current loss is: 8.501864e-6, Iteration: 1651
Current loss is: 9.06509e-6, Iteration: 1652
Current loss is: 8.511435e-6, Iteration: 1653
Current loss is: 8.954778e-6, Iteration: 1654
Current loss is: 9.025545e-6, Iteration: 1655
Current loss is: 8.616851e-6, Iteration: 1656
Current loss is: 8.624054e-6, Iteration: 1657
Current loss is: 9.087414e-6, Iteration: 1658
Current loss is: 9.058536e-6, Iteration: 1659
Current loss is: 9.091368e-6, Iteration: 1660
Current loss is: 9.032063e-6, Iteration: 1661
Current loss is: 9.085479e-6, Iteration: 1662
Current loss is: 8.98257e-6, Iteration: 1663
Current loss is: 8.992645e-6, Iteration: 1664
Current loss is: 9.087509e-6, Iteration: 1665
Current loss is: 9.008208e-6, Iteration: 1666
Current loss is: 9.104328e-6, Iteration: 1667
Current loss is: 9.030742e-6, Iteration: 1668
Current loss is: 8.573088e-6, Iteration: 1669
Current loss is: 8.984717e-6, Iteration: 1670
Current loss is: 8.988371e-6, Iteration: 1671
Current loss is: 8.979306e-6, Iteration: 1672

Current loss is: 9.061725e-6, Iteration: 1673
Current loss is: 8.560648e-6, Iteration: 1674
Current loss is: 8.976304e-6, Iteration: 1675
Current loss is: 8.709418e-6, Iteration: 1676
Current loss is: 9.006066e-6, Iteration: 1677
Current loss is: 8.95747e-6, Iteration: 1678
Current loss is: 9.0785525e-6, Iteration: 1679
Current loss is: 8.6002965e-6, Iteration: 1680
Current loss is: 8.62741e-6, Iteration: 1681
Current loss is: 9.0530875e-6, Iteration: 1682
Current loss is: 8.541771e-6, Iteration: 1683
Current loss is: 8.9360565e-6, Iteration: 1684
Current loss is: 9.008596e-6, Iteration: 1685
Current loss is: 9.057821e-6, Iteration: 1686
Current loss is: 9.078541e-6, Iteration: 1687
Current loss is: 9.059397e-6, Iteration: 1688
Current loss is: 9.008053e-6, Iteration: 1689
Current loss is: 8.4991025e-6, Iteration: 1690
Current loss is: 8.941754e-6, Iteration: 1691
Current loss is: 8.994683e-6, Iteration: 1692
Current loss is: 9.011912e-6, Iteration: 1693
Current loss is: 9.092517e-6, Iteration: 1694
Current loss is: 9.083177e-6, Iteration: 1695
Current loss is: 8.519304e-6, Iteration: 1696
Current loss is: 8.367251e-6, Iteration: 1697
Current loss is: 8.4542025e-6, Iteration: 1698
Current loss is: 8.8705065e-6, Iteration: 1699
Current loss is: 9.082956e-6, Iteration: 1700
Current loss is: 8.947576e-6, Iteration: 1701
Current loss is: 9.018291e-6, Iteration: 1702
Current loss is: 8.965848e-6, Iteration: 1703
Current loss is: 9.078414e-6, Iteration: 1704
Current loss is: 8.899408e-6, Iteration: 1705
Current loss is: 8.472631e-6, Iteration: 1706
Current loss is: 9.033146e-6, Iteration: 1707
Current loss is: 8.920276e-6, Iteration: 1708
Current loss is: 8.95218e-6, Iteration: 1709
Current loss is: 8.512179e-6, Iteration: 1710
Current loss is: 8.9655205e-6, Iteration: 1711
Current loss is: 8.521434e-6, Iteration: 1712
Current loss is: 9.068728e-6, Iteration: 1713
Current loss is: 8.448238e-6, Iteration: 1714
Current loss is: 8.943206e-6, Iteration: 1715
Current loss is: 8.961652e-6, Iteration: 1716
Current loss is: 9.008494e-6, Iteration: 1717
Current loss is: 8.492856e-6, Iteration: 1718
Current loss is: 8.994344e-6, Iteration: 1719
Current loss is: 8.998229e-6, Iteration: 1720
Current loss is: 9.0241665e-6, Iteration: 1721
Current loss is: 9.008422e-6, Iteration: 1722
Current loss is: 8.442205e-6, Iteration: 1723
Current loss is: 9.0096655e-6, Iteration: 1724
Current loss is: 8.9296855e-6, Iteration: 1725
Current loss is: 8.905548e-6, Iteration: 1726
Current loss is: 8.524692e-6, Iteration: 1727
Current loss is: 8.885607e-6, Iteration: 1728
Current loss is: 8.416358e-6, Iteration: 1729
Current loss is: 8.442899e-6, Iteration: 1730
Current loss is: 8.883889e-6, Iteration: 1731
Current loss is: 8.45481e-6, Iteration: 1732
Current loss is: 8.424337e-6, Iteration: 1733
Current loss is: 9.010217e-6, Iteration: 1734
Current loss is: 8.998765e-6, Iteration: 1735
Current loss is: 8.921566e-6, Iteration: 1736
Current loss is: 8.994402e-6, Iteration: 1737
Current loss is: 8.987713e-6, Iteration: 1738
Current loss is: 8.3906e-6, Iteration: 1739
Current loss is: 8.455437e-6, Iteration: 1740
Current loss is: 9.0596195e-6, Iteration: 1741
Current loss is: 8.484472e-6, Iteration: 1742
Current loss is: 8.8964825e-6, Iteration: 1743
Current loss is: 8.965464e-6, Iteration: 1744
Current loss is: 8.972639e-6, Iteration: 1745
Current loss is: 8.431376e-6, Iteration: 1746
Current loss is: 8.9521545e-6, Iteration: 1747
Current loss is: 8.930307e-6, Iteration: 1748
Current loss is: 8.440675e-6, Iteration: 1749
Current loss is: 8.947552e-6, Iteration: 1750
Current loss is: 8.501724e-6, Iteration: 1751
Current loss is: 8.428445e-6, Iteration: 1752
Current loss is: 8.881351e-6, Iteration: 1753
Current loss is: 8.509032e-6, Iteration: 1754
Current loss is: 8.996011e-6, Iteration: 1755
Current loss is: 8.86384e-6, Iteration: 1756
Current loss is: 8.964887e-6, Iteration: 1757
Current loss is: 8.365416e-6, Iteration: 1758
Current loss is: 8.9398245e-6, Iteration: 1759
Current loss is: 8.986465e-6, Iteration: 1760
Current loss is: 8.99216e-6, Iteration: 1761
Current loss is: 8.438105e-6, Iteration: 1762
Current loss is: 8.987889e-6, Iteration: 1763
Current loss is: 8.976678e-6, Iteration: 1764
Current loss is: 8.975187e-6, Iteration: 1765
Current loss is: 8.897694e-6, Iteration: 1766
Current loss is: 8.876674e-6, Iteration: 1767
Current loss is: 8.941755e-6, Iteration: 1768
Current loss is: 8.970944e-6, Iteration: 1769
Current loss is: 8.453839e-6, Iteration: 1770
Current loss is: 8.493015e-6, Iteration: 1771

Current loss is: 8.265289e-6, Iteration: 1772
Current loss is: 8.369196e-6, Iteration: 1773
Current loss is: 8.367135e-6, Iteration: 1774
Current loss is: 8.448334e-6, Iteration: 1775
Current loss is: 8.398588e-6, Iteration: 1776
Current loss is: 8.864721e-6, Iteration: 1777
Current loss is: 8.8984e-6, Iteration: 1778
Current loss is: 8.868556e-6, Iteration: 1779
Current loss is: 8.92103e-6, Iteration: 1780
Current loss is: 8.912344e-6, Iteration: 1781
Current loss is: 8.3255645e-6, Iteration: 1782
Current loss is: 8.485353e-6, Iteration: 1783
Current loss is: 8.400164e-6, Iteration: 1784
Current loss is: 8.799848e-6, Iteration: 1785
Current loss is: 8.389761e-6, Iteration: 1786
Current loss is: 8.278887e-6, Iteration: 1787
Current loss is: 8.36277e-6, Iteration: 1788
Current loss is: 8.897789e-6, Iteration: 1789
Current loss is: 8.873961e-6, Iteration: 1790
Current loss is: 8.359286e-6, Iteration: 1791
Current loss is: 8.901487e-6, Iteration: 1792
Current loss is: 8.346472e-6, Iteration: 1793
Current loss is: 8.911775e-6, Iteration: 1794
Current loss is: 8.909859e-6, Iteration: 1795
Current loss is: 8.362988e-6, Iteration: 1796
Current loss is: 8.986612e-6, Iteration: 1797
Current loss is: 8.394789e-6, Iteration: 1798
Current loss is: 8.317371e-6, Iteration: 1799
Current loss is: 8.816871e-6, Iteration: 1800
Current loss is: 8.811889e-6, Iteration: 1801
Current loss is: 8.96815e-6, Iteration: 1802
Current loss is: 8.8936395e-6, Iteration: 1803
Current loss is: 8.9211e-6, Iteration: 1804
Current loss is: 8.499915e-6, Iteration: 1805
Current loss is: 8.916748e-6, Iteration: 1806
Current loss is: 8.934872e-6, Iteration: 1807
Current loss is: 8.986398e-6, Iteration: 1808
Current loss is: 8.230212e-6, Iteration: 1809
Current loss is: 8.465782e-6, Iteration: 1810
Current loss is: 8.940945e-6, Iteration: 1811
Current loss is: 8.814574e-6, Iteration: 1812
Current loss is: 8.862077e-6, Iteration: 1813
Current loss is: 8.893415e-6, Iteration: 1814
Current loss is: 8.90452e-6, Iteration: 1815
Current loss is: 8.833842e-6, Iteration: 1816
Current loss is: 8.765477e-6, Iteration: 1817
Current loss is: 8.837493e-6, Iteration: 1818
Current loss is: 8.463419e-6, Iteration: 1819
Current loss is: 8.82673e-6, Iteration: 1820
Current loss is: 8.757656e-6, Iteration: 1821
Current loss is: 8.84459e-6, Iteration: 1822
Current loss is: 8.763185e-6, Iteration: 1823
Current loss is: 8.384644e-6, Iteration: 1824
Current loss is: 8.3161485e-6, Iteration: 1825
Current loss is: 8.8285e-6, Iteration: 1826
Current loss is: 8.890942e-6, Iteration: 1827
Current loss is: 8.773199e-6, Iteration: 1828
Current loss is: 8.772002e-6, Iteration: 1829
Current loss is: 8.469717e-6, Iteration: 1830
Current loss is: 8.819095e-6, Iteration: 1831
Current loss is: 8.839123e-6, Iteration: 1832
Current loss is: 8.223128e-6, Iteration: 1833
Current loss is: 8.830701e-6, Iteration: 1834
Current loss is: 8.789429e-6, Iteration: 1835
Current loss is: 8.8751e-6, Iteration: 1836
Current loss is: 8.436953e-6, Iteration: 1837
Current loss is: 8.809151e-6, Iteration: 1838
Current loss is: 8.859545e-6, Iteration: 1839
Current loss is: 8.88618e-6, Iteration: 1840
Current loss is: 8.352209e-6, Iteration: 1841
Current loss is: 8.849926e-6, Iteration: 1842
Current loss is: 8.862711e-6, Iteration: 1843
Current loss is: 8.861241e-6, Iteration: 1844
Current loss is: 8.38038e-6, Iteration: 1845
Current loss is: 8.73819e-6, Iteration: 1846
Current loss is: 8.348038e-6, Iteration: 1847
Current loss is: 8.870987e-6, Iteration: 1848
Current loss is: 8.294317e-6, Iteration: 1849
Current loss is: 8.371494e-6, Iteration: 1850
Current loss is: 8.817532e-6, Iteration: 1851
Current loss is: 8.259469e-6, Iteration: 1852
Current loss is: 8.305307e-6, Iteration: 1853
Current loss is: 8.241517e-6, Iteration: 1854
Current loss is: 8.874548e-6, Iteration: 1855
Current loss is: 8.206623e-6, Iteration: 1856
Current loss is: 8.780062e-6, Iteration: 1857
Current loss is: 8.252072e-6, Iteration: 1858
Current loss is: 8.251607e-6, Iteration: 1859
Current loss is: 8.825956e-6, Iteration: 1860
Current loss is: 8.282934e-6, Iteration: 1861
Current loss is: 8.755031e-6, Iteration: 1862
Current loss is: 8.787825e-6, Iteration: 1863
Current loss is: 8.832428e-6, Iteration: 1864
Current loss is: 8.293044e-6, Iteration: 1865
Current loss is: 8.829645e-6, Iteration: 1866
Current loss is: 8.319987e-6, Iteration: 1867
Current loss is: 8.7652e-6, Iteration: 1868
Current loss is: 8.674775e-6, Iteration: 1869
Current loss is: 8.723521e-6, Iteration: 1870
Current loss is: 8.261351e-6, Iteration: 1871
Current loss is: 8.799359e-6, Iteration: 1872
Current loss is: 8.307257e-6, Iteration: 1873
Current loss is: 8.786154e-6, Iteration: 1874

Current loss is: 8.252865e-6, Iteration: 1875
Current loss is: 8.878093e-6, Iteration: 1876
Current loss is: 8.256368e-6, Iteration: 1877
Current loss is: 8.7755e-6, Iteration: 1878
Current loss is: 8.402472e-6, Iteration: 1879
Current loss is: 8.847695e-6, Iteration: 1880
Current loss is: 8.749025e-6, Iteration: 1881
Current loss is: 8.374084e-6, Iteration: 1882
Current loss is: 8.798448e-6, Iteration: 1883
Current loss is: 8.828192e-6, Iteration: 1884
Current loss is: 8.339384e-6, Iteration: 1885
Current loss is: 8.856048e-6, Iteration: 1886
Current loss is: 8.362047e-6, Iteration: 1887
Current loss is: 8.80086e-6, Iteration: 1888
Current loss is: 8.268369e-6, Iteration: 1889
Current loss is: 8.30243e-6, Iteration: 1890
Current loss is: 8.781824e-6, Iteration: 1891
Current loss is: 8.313931e-6, Iteration: 1892
Current loss is: 8.327614e-6, Iteration: 1893
Current loss is: 8.294748e-6, Iteration: 1894
Current loss is: 8.739548e-6, Iteration: 1895
Current loss is: 8.736815e-6, Iteration: 1896
Current loss is: 8.739878e-6, Iteration: 1897
Current loss is: 8.299676e-6, Iteration: 1898
Current loss is: 8.777115e-6, Iteration: 1899
Current loss is: 8.785897e-6, Iteration: 1900
Current loss is: 8.740696e-6, Iteration: 1901
Current loss is: 8.384302e-6, Iteration: 1902
Current loss is: 8.7727585e-6, Iteration: 1903
Current loss is: 8.754216e-6, Iteration: 1904
Current loss is: 8.809505e-6, Iteration: 1905
Current loss is: 8.184651e-6, Iteration: 1906
Current loss is: 8.745595e-6, Iteration: 1907
Current loss is: 8.755291e-6, Iteration: 1908
Current loss is: 8.256762e-6, Iteration: 1909
Current loss is: 8.259556e-6, Iteration: 1910
Current loss is: 8.298568e-6, Iteration: 1911
Current loss is: 8.732397e-6, Iteration: 1912
Current loss is: 8.632542e-6, Iteration: 1913
Current loss is: 8.714595e-6, Iteration: 1914
Current loss is: 8.725032e-6, Iteration: 1915
Current loss is: 8.270212e-6, Iteration: 1916
Current loss is: 8.726654e-6, Iteration: 1917
Current loss is: 8.231365e-6, Iteration: 1918
Current loss is: 8.818151e-6, Iteration: 1919
Current loss is: 8.776096e-6, Iteration: 1920
Current loss is: 8.227473e-6, Iteration: 1921
Current loss is: 8.69315e-6, Iteration: 1922
Current loss is: 8.680945e-6, Iteration: 1923
Current loss is: 8.780254e-6, Iteration: 1924
Current loss is: 8.292863e-6, Iteration: 1925
Current loss is: 8.702513e-6, Iteration: 1926
Current loss is: 8.6923565e-6, Iteration: 1927
Current loss is: 8.299328e-6, Iteration: 1928
Current loss is: 8.692561e-6, Iteration: 1929
Current loss is: 8.750794e-6, Iteration: 1930
Current loss is: 8.178486e-6, Iteration: 1931
Current loss is: 8.284572e-6, Iteration: 1932
Current loss is: 8.189964e-6, Iteration: 1933
Current loss is: 8.788673e-6, Iteration: 1934
Current loss is: 8.1073085e-6, Iteration: 1935
Current loss is: 8.369967e-6, Iteration: 1936
Current loss is: 8.658166e-6, Iteration: 1937
Current loss is: 8.2450915e-6, Iteration: 1938
Current loss is: 8.088638e-6, Iteration: 1939
Current loss is: 8.218192e-6, Iteration: 1940
Current loss is: 8.292762e-6, Iteration: 1941
Current loss is: 8.278699e-6, Iteration: 1942
Current loss is: 8.752191e-6, Iteration: 1943
Current loss is: 8.69535e-6, Iteration: 1944
Current loss is: 8.223867e-6, Iteration: 1945
Current loss is: 8.154378e-6, Iteration: 1946
Current loss is: 8.700597e-6, Iteration: 1947
Current loss is: 8.272622e-6, Iteration: 1948
Current loss is: 8.215342e-6, Iteration: 1949
Current loss is: 8.164875e-6, Iteration: 1950
Current loss is: 8.658775e-6, Iteration: 1951
Current loss is: 8.172192e-6, Iteration: 1952
Current loss is: 8.668457e-6, Iteration: 1953
Current loss is: 8.699375e-6, Iteration: 1954
Current loss is: 8.684551e-6, Iteration: 1955
Current loss is: 8.686164e-6, Iteration: 1956
Current loss is: 8.704086e-6, Iteration: 1957
Current loss is: 8.223205e-6, Iteration: 1958
Current loss is: 8.745677e-6, Iteration: 1959
Current loss is: 8.083951e-6, Iteration: 1960
Current loss is: 8.718041e-6, Iteration: 1961
Current loss is: 8.164959e-6, Iteration: 1962
Current loss is: 8.78817e-6, Iteration: 1963
Current loss is: 8.735915e-6, Iteration: 1964
Current loss is: 8.572563e-6, Iteration: 1965
Current loss is: 8.686825e-6, Iteration: 1966
Current loss is: 8.800503e-6, Iteration: 1967
Current loss is: 8.707699e-6, Iteration: 1968
Current loss is: 8.678783e-6, Iteration: 1969
Current loss is: 8.609219e-6, Iteration: 1970
Current loss is: 8.148245e-6, Iteration: 1971
Current loss is: 8.727344e-6, Iteration: 1972
Current loss is: 8.201478e-6, Iteration: 1973
Current loss is: 8.102199e-6, Iteration: 1974
Current loss is: 8.121641e-6, Iteration: 1975
Current loss is: 8.095174e-6, Iteration: 1976
Current loss is: 8.682457e-6, Iteration: 1977
Current loss is: 8.205601e-6, Iteration: 1978
Current loss is: 8.12528e-6, Iteration: 1979
Current loss is: 8.59207e-6, Iteration: 1980
Current loss is: 8.03897e-6, Iteration: 1981
Current loss is: 8.172155e-6, Iteration: 1982
Current loss is: 8.181356e-6, Iteration: 1983
Current loss is: 8.093849e-6, Iteration: 1984
Current loss is: 8.593237e-6, Iteration: 1985
Current loss is: 8.177595e-6, Iteration: 1986
Current loss is: 8.607321e-6, Iteration: 1987
Current loss is: 8.192283e-6, Iteration: 1988

Current loss is: 8.673346e-6, Iteration: 1989
Current loss is: 8.089627e-6, Iteration: 1990
Current loss is: 8.628117e-6, Iteration: 1991
Current loss is: 8.0999325e-6, Iteration: 1992
Current loss is: 8.644596e-6, Iteration: 1993
Current loss is: 8.147517e-6, Iteration: 1994
Current loss is: 8.654279e-6, Iteration: 1995
Current loss is: 8.248249e-6, Iteration: 1996
Current loss is: 8.654459e-6, Iteration: 1997
Current loss is: 8.071769e-6, Iteration: 1998
Current loss is: 8.2639335e-6, Iteration: 1999
Current loss is: 8.144181e-6, Iteration: 2000
Current loss is: 8.03897e-6, Iteration: 2001

retcode: Success
Interpolation: Trained neural network interpolation
t: 0.0:0.01:1.0
u: 101-element Vector{Float64}:
  0.0
  0.00944402213521234
  0.018817099250277716
  0.028100628704810757
  0.0372748440818813
  0.04631881431473818
  0.05521045415817861
  0.06392654761776762
  0.07244278599223654
  0.08073382219436322
  0.0887733429860173
  0.09653416068613183
  0.10398832577841613
  ⋮
 -0.10296169296570452
 -0.09577268539225432
 -0.08815974116293056
 -0.08013165631047155
 -0.07169722175587259
 -0.06286520663555657
 -0.053644343975717865
 -0.04404331845383631
 -0.03407075600697663
 -0.02373521506586277
 -0.013045179212622202
 -0.002009051078320445

Comparing to the regular solver

sol2 = solve(prob, Tsit5(), saveat=sol.t)

using LinearAlgebra
norm(sol.u .- sol2.u, Inf)

0.004960657473632363