Tutorial - Implementing a custom analysis block in Lightwood

Introduction

As you might already know, Lightwood is designed to be a flexible machine learning (ML) library that is able to abstract and automate the entire ML pipeline. Crucially, it is also designed to be extended or modified very easily according to your needs, essentially offering the entire spectrum between fully automated AutoML and a lightweight wrapper for customized ML pipelines.

As such, we can identify several different customizable “phases” in the process. The relevant phase for this tutorial is the “analysis” that comes after a predictor has been trained. The goal of this phase is to generate useful insights, like accuracy metrics, confusion matrices, feature importance, etc. These particular examples are all included in the core analysis procedure that Lightwood executes.

However, the analysis procedure is structured into a sequential execution of “analysis blocks”. Each analysis block should generate a well-defined set of insights, as well as handling any actions regarding these at inference time.

As an example, one of the core blocks is the Inductive Conformal Prediction (ICP) block, which handles the confidence estimation of all Lightwood predictors. The logic within can be complex at times, but thanks to the block abstraction we can deal with it in a structured manner. As this ICP block is used when generating predictions, it implements the two main methods that the BaseAnalysisBlock class specifies: .analyze() to setup everything that is needed, and .explain() to actually estimate the confidence in any given prediction.

Objective

In this tutorial, we will go through the steps required to implement your own analysis blocks to customize the insights of any Lightwood predictor!

In particular, we will implement a “model correlation heatmap” block: we want to compare the predictions of all mixers inside a BestOf ensemble object, to understand how they might differ in their overall behavior.

[1]:
from typing import Dict, Tuple
import pandas as pd
import lightwood
lightwood.__version__


INFO:lightwood-2822:No torchvision detected, image helpers not supported.
INFO:lightwood-2822:No torchvision/pillow detected, image encoder not supported
[1]:
'22.6.1.2'

Step 1: figuring out what we need

When designing an analysis block, an important choice needs to be made: will this block operate when calling the predictor? Or is it only going to describe its performance once in the held-out validation dataset?

Being in the former case means we need to implement both .analyze() and .explain() methods, while the latter case only needs an .analyze() method. Our ModelCorrelationHeatmap belongs to this second category.

Let’s start the implementation by inheriting from BaseAnalysisBlock:

[2]:
from lightwood.analysis import BaseAnalysisBlock

class ModelCorrelationHeatmap(BaseAnalysisBlock):
    def __init__(self):
        super().__init__()

    def analyze(self, info: Dict[str, object], **kwargs) -> Dict[str, object]:
        return info

    def explain(self,
                row_insights: pd.DataFrame,
                global_insights: Dict[str, object], **kwargs) -> Tuple[pd.DataFrame, Dict[str, object]]:

        return row_insights, global_insights
[3]:
ModelCorrelationHeatmap()
[3]:
<__main__.ModelCorrelationHeatmap at 0x7efc27ed6c40>

Right now, our newly created analysis block doesn’t do much, apart from returning the info and insights (row_insights and global_insights) exactly as it received them from the previous block.

As previously discussed, we only need to implement a procedure that runs post-training, no action is required at inference time. This means we can use the default .explain() behavior in the parent class:

[4]:
class ModelCorrelationHeatmap(BaseAnalysisBlock):
    def __init__(self):
        super().__init__()

    def analyze(self, info: Dict[str, object], **kwargs) -> Dict[str, object]:
        return info

Step 2: Implementing the custom analysis block

Okay, now for the fun bit: we have to implement a correlation heatmap between the predictions of all mixers inside a BestOf ensemble. This is currently the only ensemble implemented in Lightwood, but it is a good idea to explicitly check that the type of the ensemble is what we expect.

A natural question to ask at this point is: what information do we have to implement the procedure? You’ll note that, apart from the info dictionary, we receive a kwargs dictionary. You can check out the full documentation for more details, but the keys (and respective value types) exposed in this object by default are:

[5]:
kwargs = {
        'predictor': 'lightwood.ensemble.BaseEnsemble',
        'target': 'str',
        'input_cols': 'list',
        'dtype_dict': 'dict',
        'normal_predictions': 'pd.DataFrame',
        'data': 'pd.DataFrame',
        'train_data': 'lightwood.data.encoded_ds.EncodedDs',
        'encoded_val_data': 'lightwood.data.encoded_ds.EncodedDs',
        'is_classification': 'bool',
        'is_numerical': 'bool',
        'is_multi_ts': 'bool',
        'stats_info': 'lightwood.api.types.StatisticalAnalysis',
        'ts_cfg': 'lightwood.api.types.TimeseriesSettings',
        'accuracy_functions': 'list',
        'has_pretrained_text_enc': 'bool'
}

As you can see there is lots to work with, but for this example we will focus on using:

  1. The predictor ensemble

  2. The encoded_val_data to generate predictions for each mixer inside the ensemble

And the insight we’re want to produce is a matrix that compares the output of all mixers and computes the correlation between them.

Let’s implement the algorithm:

[6]:
%%writefile model_correlation.py

from typing import Dict
from types import SimpleNamespace

import numpy as np

from lightwood.ensemble import BestOf
from lightwood.analysis import BaseAnalysisBlock


class ModelCorrelationHeatmap(BaseAnalysisBlock):
    def __init__(self):
        super().__init__()

    def analyze(self, info: Dict[str, object], **kwargs) -> Dict[str, object]:
        ns = SimpleNamespace(**kwargs)

        # only triggered with the right type of ensemble
        if isinstance(ns.predictor, BestOf):

            # store prediction from every mixer
            all_predictions = []

            for mixer in ns.predictor.mixers:
                predictions = mixer(ns.encoded_val_data).values  # retrieve np.ndarray from the returned pd.DataFrame
                all_predictions.append(predictions.flatten().astype(int))  # flatten and cast labels to int

            # calculate correlation matrix
            corrs = np.corrcoef(np.array(all_predictions))

            # save inside `info` object
            info['mixer_correlation'] = corrs

        return info

Writing model_correlation.py

Notice the use of SimpleNamespace for dot notation accessors.

The procedure above is fairly straightforward, as we leverage numpy’s corrcoef() function to generate the matrix.

Finally, it is very important to add the output to info so that it is saved inside the actual predictor object.

Step 3: Exposing the block to Lightwood

To use this in an arbitrary script, we need to add the above class (and all necessary imports) to a .py file inside one of the following directories:

  • ~/lightwood_modules (where ~ is your home directory, e.g. /Users/username/ for macOS and /home/username/ for linux

  • /etc/lightwood_modules

Lightwood will scan these directories and import any class so that they can be found and used by the JsonAI code generating module.

To continue, please save the code cell above as ``model_correlation.py`` in one of the indicated directories.

Step 4: Final test run

Ok! Everything looks set to try out our custom block. Let’s generate a predictor for this sample dataset, and see whether our new insights are any good.

First, it is important to add our ModelCorrelationHeatmap to the analysis_blocks attribute of the Json AI object that will generate your predictor code.

[7]:
from lightwood.api.high_level import ProblemDefinition, json_ai_from_problem, load_custom_module

# First, load the custom module we wrote
load_custom_module('model_correlation.py')

# read dataset
df = pd.read_csv('https://raw.githubusercontent.com/mindsdb/lightwood/stable/tests/data/hdi.csv')

# define the predictive task
pdef = ProblemDefinition.from_dict({
    'target': 'Development Index',         # column you want to predict
    'time_aim': 100,
})

# generate the Json AI intermediate representation from the data and its corresponding settings
json_ai = json_ai_from_problem(df, problem_definition=pdef)

# add the custom list of analysis blocks; in this case, composed of a single block
json_ai.analysis_blocks = [{
    'module': 'model_correlation.ModelCorrelationHeatmap',
    'args': {}
}]
INFO:lightwood-2822:Analyzing a sample of 222
INFO:lightwood-2822:from a total population of 225, this is equivalent to 98.7% of your data.
INFO:lightwood-2822:Infering type for: Population
INFO:lightwood-2822:Column Population has data type integer
INFO:lightwood-2822:Infering type for: Area (sq. mi.)
INFO:lightwood-2822:Column Area (sq. mi.) has data type integer
INFO:lightwood-2822:Infering type for: Pop. Density 
INFO:lightwood-2822:Column Pop. Density  has data type float
INFO:lightwood-2822:Infering type for: GDP ($ per capita)
INFO:lightwood-2822:Column GDP ($ per capita) has data type integer
INFO:lightwood-2822:Infering type for: Literacy (%)
INFO:lightwood-2822:Column Literacy (%) has data type float
INFO:lightwood-2822:Infering type for: Infant mortality 
INFO:lightwood-2822:Column Infant mortality  has data type float
INFO:lightwood-2822:Infering type for: Development Index
INFO:lightwood-2822:Column Development Index has data type categorical
INFO:lightwood-2822:Starting statistical analysis
INFO:lightwood-2822:Finished statistical analysis

We can take a look at the respective Json AI key just to confirm our newly added analysis block is in there:

[8]:
json_ai.analysis_blocks
[8]:
[{'module': 'model_correlation.ModelCorrelationHeatmap', 'args': {}}]

Now we are ready to create a predictor from this Json AI, and subsequently train it:

[9]:
from lightwood.api.high_level import code_from_json_ai, predictor_from_code

code = code_from_json_ai(json_ai)
predictor = predictor_from_code(code)

predictor.learn(df)
INFO:lightwood-2822:Unable to import black formatter, predictor code might be a bit ugly.
INFO:lightwood-2822:[Learn phase 1/8] - Statistical analysis
INFO:lightwood-2822:Starting statistical analysis
INFO:lightwood-2822:Finished statistical analysis
DEBUG:lightwood-2822: `analyze_data` runtime: 0.02 seconds
INFO:lightwood-2822:[Learn phase 2/8] - Data preprocessing
INFO:lightwood-2822:Cleaning the data
DEBUG:lightwood-2822: `preprocess` runtime: 0.01 seconds
INFO:lightwood-2822:[Learn phase 3/8] - Data splitting
INFO:lightwood-2822:Splitting the data into train/test
DEBUG:lightwood-2822: `split` runtime: 0.04 seconds
INFO:lightwood-2822:[Learn phase 4/8] - Preparing encoders
INFO:lightwood-2822:Done running for: Population
INFO:lightwood-2822:Done running for: Area (sq. mi.)
INFO:lightwood-2822:Done running for: Pop. Density 
INFO:lightwood-2822:Done running for: GDP ($ per capita)
INFO:lightwood-2822:Done running for: Literacy (%)
INFO:lightwood-2822:Done running for: Infant mortality 
INFO:lightwood-2822:Encoding UNKNOWN categories as index 0
DEBUG:lightwood-2822: `prepare` runtime: 0.16 seconds
INFO:lightwood-2822:[Learn phase 5/8] - Feature generation
INFO:lightwood-2822:Featurizing the data
DEBUG:lightwood-2822: `featurize` runtime: 0.0 seconds
INFO:lightwood-2822:[Learn phase 6/8] - Mixer training
INFO:lightwood-2822:Training the mixers
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:177: UserWarning: Found `num_iterations` in params. Will use it instead of argument
  _log_warning(f"Found `{alias}` in params. Will use it instead of argument")
[LightGBM] [Fatal] GPU Tree Learner was not enabled in this build.
Please recompile with CMake option -DUSE_GPU=1
WARNING:lightwood-2822:LightGBM running on CPU, this somewhat slower than the GPU version, consider using a GPU instead
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:115: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
  warnings.warn("torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.")
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/pytorch_ranger/ranger.py:172: UserWarning: This overload of addcmul_ is deprecated:
        addcmul_(Number value, Tensor tensor1, Tensor tensor2)
Consider using one of the following signatures instead:
        addcmul_(Tensor tensor1, Tensor tensor2, *, Number value) (Triggered internally at  ../torch/csrc/utils/python_arg_parser.cpp:1025.)
  exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad)
INFO:lightwood-2822:Loss of 2.2819137573242188 with learning rate 0.0001
INFO:lightwood-2822:Loss of 2.268077850341797 with learning rate 0.00014
INFO:lightwood-2822:Loss of 2.224043846130371 with learning rate 0.00019599999999999997
INFO:lightwood-2822:Loss of 1.7930376529693604 with learning rate 0.00027439999999999995
INFO:lightwood-2822:Loss of 2.3550076484680176 with learning rate 0.0003841599999999999
INFO:lightwood-2822:Found learning rate of: 0.00027439999999999995
INFO:lightwood-2822:Loss @ epoch 1: 1.8338336944580078
INFO:lightwood-2822:Loss @ epoch 2: 1.939591646194458
INFO:lightwood-2822:Loss @ epoch 3: 1.9169481992721558
INFO:lightwood-2822:Loss @ epoch 4: 1.8937150239944458
INFO:lightwood-2822:Loss @ epoch 5: 1.86952543258667
INFO:lightwood-2822:Loss @ epoch 6: 1.8448940515518188
INFO:lightwood-2822:Loss @ epoch 7: 1.7847262620925903
INFO:lightwood-2822:Loss @ epoch 8: 1.7597827911376953
INFO:lightwood-2822:Loss @ epoch 9: 1.736220359802246
INFO:lightwood-2822:Loss @ epoch 10: 1.714125394821167
INFO:lightwood-2822:Loss @ epoch 11: 1.6933625936508179
INFO:lightwood-2822:Loss @ epoch 12: 1.6740318536758423
INFO:lightwood-2822:Loss @ epoch 13: 1.631773591041565
INFO:lightwood-2822:Loss @ epoch 14: 1.616595983505249
INFO:lightwood-2822:Loss @ epoch 15: 1.6023756265640259
INFO:lightwood-2822:Loss @ epoch 16: 1.5890617370605469
INFO:lightwood-2822:Loss @ epoch 17: 1.576419711112976
INFO:lightwood-2822:Loss @ epoch 18: 1.564160943031311
INFO:lightwood-2822:Loss @ epoch 19: 1.5355868339538574
INFO:lightwood-2822:Loss @ epoch 20: 1.5242055654525757
INFO:lightwood-2822:Loss @ epoch 21: 1.5129410028457642
INFO:lightwood-2822:Loss @ epoch 22: 1.501926064491272
INFO:lightwood-2822:Loss @ epoch 23: 1.4911010265350342
INFO:lightwood-2822:Loss @ epoch 24: 1.4801454544067383
INFO:lightwood-2822:Loss @ epoch 25: 1.4535719156265259
INFO:lightwood-2822:Loss @ epoch 26: 1.443029761314392
INFO:lightwood-2822:Loss @ epoch 27: 1.432600975036621
INFO:lightwood-2822:Loss @ epoch 28: 1.4224669933319092
INFO:lightwood-2822:Loss @ epoch 29: 1.4125334024429321
INFO:lightwood-2822:Loss @ epoch 30: 1.4025007486343384
INFO:lightwood-2822:Loss @ epoch 31: 1.3782844543457031
INFO:lightwood-2822:Loss @ epoch 32: 1.3688035011291504
INFO:lightwood-2822:Loss @ epoch 33: 1.3593981266021729
INFO:lightwood-2822:Loss @ epoch 34: 1.350266933441162
INFO:lightwood-2822:Loss @ epoch 35: 1.341304898262024
INFO:lightwood-2822:Loss @ epoch 36: 1.3322371244430542
INFO:lightwood-2822:Loss @ epoch 37: 1.310435175895691
INFO:lightwood-2822:Loss @ epoch 38: 1.301945686340332
INFO:lightwood-2822:Loss @ epoch 39: 1.2935556173324585
INFO:lightwood-2822:Loss @ epoch 40: 1.2854571342468262
INFO:lightwood-2822:Loss @ epoch 41: 1.2775602340698242
INFO:lightwood-2822:Loss @ epoch 42: 1.2695804834365845
INFO:lightwood-2822:Loss @ epoch 43: 1.2506136894226074
INFO:lightwood-2822:Loss @ epoch 44: 1.2433480024337769
INFO:lightwood-2822:Loss @ epoch 45: 1.2362172603607178
INFO:lightwood-2822:Loss @ epoch 46: 1.2293823957443237
INFO:lightwood-2822:Loss @ epoch 47: 1.2227636575698853
INFO:lightwood-2822:Loss @ epoch 48: 1.2160584926605225
INFO:lightwood-2822:Loss @ epoch 49: 1.2002322673797607
INFO:lightwood-2822:Loss @ epoch 50: 1.1941956281661987
INFO:lightwood-2822:Loss @ epoch 51: 1.1882643699645996
INFO:lightwood-2822:Loss @ epoch 52: 1.1825472116470337
INFO:lightwood-2822:Loss @ epoch 53: 1.1769970655441284
INFO:lightwood-2822:Loss @ epoch 54: 1.171310544013977
INFO:lightwood-2822:Loss @ epoch 55: 1.1578630208969116
INFO:lightwood-2822:Loss @ epoch 56: 1.1526927947998047
INFO:lightwood-2822:Loss @ epoch 57: 1.1476008892059326
INFO:lightwood-2822:Loss @ epoch 58: 1.1426829099655151
INFO:lightwood-2822:Loss @ epoch 59: 1.13791823387146
INFO:lightwood-2822:Loss @ epoch 60: 1.1329773664474487
INFO:lightwood-2822:Loss @ epoch 61: 1.121259093284607
INFO:lightwood-2822:Loss @ epoch 62: 1.1167134046554565
INFO:lightwood-2822:Loss @ epoch 63: 1.112216591835022
INFO:lightwood-2822:Loss @ epoch 64: 1.107849359512329
INFO:lightwood-2822:Loss @ epoch 65: 1.1036131381988525
INFO:lightwood-2822:Loss @ epoch 66: 1.099169135093689
INFO:lightwood-2822:Loss @ epoch 67: 1.0885926485061646
INFO:lightwood-2822:Loss @ epoch 68: 1.0844480991363525
INFO:lightwood-2822:Loss @ epoch 69: 1.0803428888320923
INFO:lightwood-2822:Loss @ epoch 70: 1.0763431787490845
INFO:lightwood-2822:Loss @ epoch 71: 1.0724674463272095
INFO:lightwood-2822:Loss @ epoch 72: 1.0683684349060059
INFO:lightwood-2822:Loss @ epoch 73: 1.0586202144622803
INFO:lightwood-2822:Loss @ epoch 74: 1.0547850131988525
INFO:lightwood-2822:Loss @ epoch 75: 1.0509812831878662
INFO:lightwood-2822:Loss @ epoch 76: 1.0472744703292847
INFO:lightwood-2822:Loss @ epoch 77: 1.0436888933181763
INFO:lightwood-2822:Loss @ epoch 78: 1.03987717628479
INFO:lightwood-2822:Loss @ epoch 79: 1.0308085680007935
INFO:lightwood-2822:Loss @ epoch 80: 1.0272281169891357
INFO:lightwood-2822:Loss @ epoch 81: 1.0236787796020508
INFO:lightwood-2822:Loss @ epoch 82: 1.0202243328094482
INFO:lightwood-2822:Loss @ epoch 83: 1.0168993473052979
INFO:lightwood-2822:Loss @ epoch 84: 1.0133183002471924
INFO:lightwood-2822:Loss @ epoch 85: 1.0047935247421265
INFO:lightwood-2822:Loss @ epoch 86: 1.0014185905456543
INFO:lightwood-2822:Loss @ epoch 87: 0.9980721473693848
INFO:lightwood-2822:Loss @ epoch 88: 0.9948106408119202
INFO:lightwood-2822:Loss @ epoch 89: 0.9916837811470032
INFO:lightwood-2822:Loss @ epoch 90: 0.9882921576499939
INFO:lightwood-2822:Loss @ epoch 91: 0.9802191853523254
INFO:lightwood-2822:Loss @ epoch 92: 0.9770156145095825
INFO:lightwood-2822:Loss @ epoch 93: 0.9738380908966064
INFO:lightwood-2822:Loss @ epoch 94: 0.9707422852516174
INFO:lightwood-2822:Loss @ epoch 95: 0.9677847027778625
INFO:lightwood-2822:Loss @ epoch 96: 0.9645572304725647
INFO:lightwood-2822:Loss @ epoch 97: 0.9569082856178284
INFO:lightwood-2822:Loss @ epoch 98: 0.9538640379905701
INFO:lightwood-2822:Loss @ epoch 99: 0.950847864151001
INFO:lightwood-2822:Loss @ epoch 100: 0.9479117393493652
INFO:lightwood-2822:Loss @ epoch 101: 0.945122241973877
INFO:lightwood-2822:Loss @ epoch 102: 0.9420557618141174
INFO:lightwood-2822:Loss @ epoch 103: 0.9347626566886902
INFO:lightwood-2822:Loss @ epoch 104: 0.931851327419281
INFO:lightwood-2822:Loss @ epoch 105: 0.9289681911468506
INFO:lightwood-2822:Loss @ epoch 106: 0.9261629581451416
INFO:lightwood-2822:Loss @ epoch 107: 0.9235121607780457
INFO:lightwood-2822:Loss @ epoch 108: 0.9205791354179382
INFO:lightwood-2822:Loss @ epoch 109: 0.9136189818382263
INFO:lightwood-2822:Loss @ epoch 110: 0.9108605980873108
INFO:lightwood-2822:Loss @ epoch 111: 0.9081340432167053
INFO:lightwood-2822:Loss @ epoch 112: 0.9054849743843079
INFO:lightwood-2822:Loss @ epoch 113: 0.9029994606971741
INFO:lightwood-2822:Loss @ epoch 114: 0.9002260565757751
INFO:lightwood-2822:Loss @ epoch 115: 0.8936280012130737
INFO:lightwood-2822:Loss @ epoch 116: 0.8909881114959717
INFO:lightwood-2822:Loss @ epoch 117: 0.8883811831474304
INFO:lightwood-2822:Loss @ epoch 118: 0.8858554363250732
INFO:lightwood-2822:Loss @ epoch 119: 0.8835002779960632
INFO:lightwood-2822:Loss @ epoch 120: 0.8808587789535522
INFO:lightwood-2822:Loss @ epoch 121: 0.8745760917663574
INFO:lightwood-2822:Loss @ epoch 122: 0.8720563054084778
INFO:lightwood-2822:Loss @ epoch 123: 0.8695646524429321
INFO:lightwood-2822:Loss @ epoch 124: 0.8671508431434631
INFO:lightwood-2822:Loss @ epoch 125: 0.8649152517318726
INFO:lightwood-2822:Loss @ epoch 126: 0.8623828887939453
INFO:lightwood-2822:Loss @ epoch 127: 0.8563553690910339
INFO:lightwood-2822:Loss @ epoch 128: 0.853931725025177
INFO:lightwood-2822:Loss @ epoch 129: 0.8515380024909973
INFO:lightwood-2822:Loss @ epoch 130: 0.849219560623169
INFO:lightwood-2822:Loss @ epoch 131: 0.847082793712616
INFO:lightwood-2822:Loss @ epoch 132: 0.8446438312530518
INFO:lightwood-2822:Loss @ epoch 133: 0.8388401865959167
INFO:lightwood-2822:Loss @ epoch 134: 0.8364961743354797
INFO:lightwood-2822:Loss @ epoch 135: 0.8341687321662903
INFO:lightwood-2822:Loss @ epoch 136: 0.8319140076637268
INFO:lightwood-2822:Loss @ epoch 137: 0.8298398852348328
INFO:lightwood-2822:Loss @ epoch 138: 0.8274626731872559
INFO:lightwood-2822:Loss @ epoch 139: 0.8217914700508118
INFO:lightwood-2822:Loss @ epoch 140: 0.8195037841796875
INFO:lightwood-2822:Loss @ epoch 141: 0.817251443862915
INFO:lightwood-2822:Loss @ epoch 142: 0.8150755763053894
INFO:lightwood-2822:Loss @ epoch 143: 0.8130951523780823
INFO:lightwood-2822:Loss @ epoch 144: 0.8108086585998535
INFO:lightwood-2822:Loss @ epoch 145: 0.8053432106971741
INFO:lightwood-2822:Loss @ epoch 146: 0.8031229376792908
INFO:lightwood-2822:Loss @ epoch 147: 0.8009352087974548
INFO:lightwood-2822:Loss @ epoch 148: 0.7988256216049194
INFO:lightwood-2822:Loss @ epoch 149: 0.7969179153442383
INFO:lightwood-2822:Loss @ epoch 150: 0.7947056889533997
INFO:lightwood-2822:Loss @ epoch 151: 0.7893959283828735
INFO:lightwood-2822:Loss @ epoch 152: 0.7872317433357239
INFO:lightwood-2822:Loss @ epoch 153: 0.7850971817970276
INFO:lightwood-2822:Loss @ epoch 154: 0.7830415964126587
INFO:lightwood-2822:Loss @ epoch 155: 0.7811943292617798
INFO:lightwood-2822:Loss @ epoch 156: 0.7790356278419495
INFO:lightwood-2822:Loss @ epoch 157: 0.7738635540008545
INFO:lightwood-2822:Loss @ epoch 158: 0.7717564702033997
INFO:lightwood-2822:Loss @ epoch 159: 0.7696824073791504
INFO:lightwood-2822:Loss @ epoch 160: 0.7676904201507568
INFO:lightwood-2822:Loss @ epoch 161: 0.7659105062484741
INFO:lightwood-2822:Loss @ epoch 162: 0.763818621635437
INFO:lightwood-2822:Loss @ epoch 163: 0.7588075995445251
INFO:lightwood-2822:Loss @ epoch 164: 0.7567644119262695
INFO:lightwood-2822:Loss @ epoch 165: 0.7547507882118225
INFO:lightwood-2822:Loss @ epoch 166: 0.7528188824653625
INFO:lightwood-2822:Loss @ epoch 167: 0.7511033415794373
INFO:lightwood-2822:Loss @ epoch 168: 0.7490715384483337
INFO:lightwood-2822:Loss @ epoch 169: 0.7442001104354858
INFO:lightwood-2822:Loss @ epoch 170: 0.7422056794166565
INFO:lightwood-2822:Loss @ epoch 171: 0.7402476668357849
INFO:lightwood-2822:Loss @ epoch 172: 0.7383757829666138
INFO:lightwood-2822:Loss @ epoch 173: 0.7367260456085205
INFO:lightwood-2822:Loss @ epoch 174: 0.7347584366798401
INFO:lightwood-2822:Loss @ epoch 175: 0.7300329208374023
INFO:lightwood-2822:Loss @ epoch 176: 0.728087306022644
INFO:lightwood-2822:Loss @ epoch 177: 0.7261723875999451
INFO:lightwood-2822:Loss @ epoch 178: 0.7243432998657227
INFO:lightwood-2822:Loss @ epoch 179: 0.7227376103401184
INFO:lightwood-2822:Loss @ epoch 180: 0.7208092212677002
INFO:lightwood-2822:Loss @ epoch 181: 0.7161564826965332
INFO:lightwood-2822:Loss @ epoch 182: 0.7142320871353149
INFO:lightwood-2822:Loss @ epoch 183: 0.7123438119888306
INFO:lightwood-2822:Loss @ epoch 184: 0.7105300426483154
INFO:lightwood-2822:Loss @ epoch 185: 0.7089465856552124
INFO:lightwood-2822:Loss @ epoch 186: 0.707041323184967
INFO:lightwood-2822:Loss @ epoch 187: 0.7024451494216919
INFO:lightwood-2822:Loss @ epoch 188: 0.7005430459976196
INFO:lightwood-2822:Loss @ epoch 189: 0.698675274848938
INFO:lightwood-2822:Loss @ epoch 190: 0.696900486946106
INFO:lightwood-2822:Loss @ epoch 191: 0.6953522562980652
INFO:lightwood-2822:Loss @ epoch 192: 0.6934759616851807
INFO:lightwood-2822:Loss @ epoch 193: 0.6889737844467163
INFO:lightwood-2822:Loss @ epoch 194: 0.687127947807312
INFO:lightwood-2822:Loss @ epoch 195: 0.6853359937667847
INFO:lightwood-2822:Loss @ epoch 196: 0.6836408972740173
INFO:lightwood-2822:Loss @ epoch 197: 0.6821861267089844
INFO:lightwood-2822:Loss @ epoch 198: 0.6804043650627136
INFO:lightwood-2822:Loss @ epoch 199: 0.6761071085929871
INFO:lightwood-2822:Loss @ epoch 200: 0.6743407845497131
INFO:lightwood-2822:Loss @ epoch 201: 0.6726114749908447
INFO:lightwood-2822:Loss @ epoch 202: 0.6709753274917603
INFO:lightwood-2822:Loss @ epoch 203: 0.6695717573165894
INFO:lightwood-2822:Loss @ epoch 204: 0.6678540110588074
INFO:lightwood-2822:Loss @ epoch 205: 0.6637123823165894
INFO:lightwood-2822:Loss @ epoch 206: 0.6619884371757507
INFO:lightwood-2822:Loss @ epoch 207: 0.6603102684020996
INFO:lightwood-2822:Loss @ epoch 208: 0.6587331295013428
INFO:lightwood-2822:Loss @ epoch 209: 0.657393217086792
INFO:lightwood-2822:Loss @ epoch 210: 0.6557157039642334
INFO:lightwood-2822:Loss @ epoch 211: 0.6516337394714355
INFO:lightwood-2822:Loss @ epoch 212: 0.6499212384223938
INFO:lightwood-2822:Loss @ epoch 213: 0.6482642889022827
INFO:lightwood-2822:Loss @ epoch 214: 0.6467219591140747
INFO:lightwood-2822:Loss @ epoch 215: 0.6454133987426758
INFO:lightwood-2822:Loss @ epoch 216: 0.6437739729881287
INFO:lightwood-2822:Loss @ epoch 217: 0.6397921442985535
INFO:lightwood-2822:Loss @ epoch 218: 0.6381191611289978
INFO:lightwood-2822:Loss @ epoch 219: 0.6364969611167908
INFO:lightwood-2822:Loss @ epoch 220: 0.6349803805351257
INFO:lightwood-2822:Loss @ epoch 221: 0.6336992383003235
INFO:lightwood-2822:Loss @ epoch 222: 0.6320872902870178
INFO:lightwood-2822:Loss @ epoch 223: 0.6281730532646179
INFO:lightwood-2822:Loss @ epoch 224: 0.6265391707420349
INFO:lightwood-2822:Loss @ epoch 225: 0.6249567866325378
INFO:lightwood-2822:Loss @ epoch 226: 0.6234968900680542
INFO:lightwood-2822:Loss @ epoch 227: 0.6222765445709229
INFO:lightwood-2822:Loss @ epoch 228: 0.620727002620697
INFO:lightwood-2822:Loss @ epoch 229: 0.6170168519020081
INFO:lightwood-2822:Loss @ epoch 230: 0.6154745221138
INFO:lightwood-2822:Loss @ epoch 231: 0.6139799952507019
INFO:lightwood-2822:Loss @ epoch 232: 0.6126092076301575
INFO:lightwood-2822:Loss @ epoch 233: 0.6114785671234131
INFO:lightwood-2822:Loss @ epoch 234: 0.6100220084190369
INFO:lightwood-2822:Loss @ epoch 235: 0.6064783930778503
INFO:lightwood-2822:Loss @ epoch 236: 0.6049812436103821
INFO:lightwood-2822:Loss @ epoch 237: 0.6035362482070923
INFO:lightwood-2822:Loss @ epoch 238: 0.6022168397903442
INFO:lightwood-2822:Loss @ epoch 239: 0.6011382341384888
INFO:lightwood-2822:Loss @ epoch 240: 0.5997297763824463
INFO:lightwood-2822:Loss @ epoch 241: 0.5962937474250793
INFO:lightwood-2822:Loss @ epoch 242: 0.5948315262794495
INFO:lightwood-2822:Loss @ epoch 243: 0.5934163331985474
INFO:lightwood-2822:Loss @ epoch 244: 0.5921291708946228
INFO:lightwood-2822:Loss @ epoch 245: 0.5910803079605103
INFO:lightwood-2822:Loss @ epoch 246: 0.5897043943405151
INFO:lightwood-2822:Loss @ epoch 247: 0.586400032043457
INFO:lightwood-2822:Loss @ epoch 248: 0.5849912166595459
INFO:lightwood-2822:Loss @ epoch 249: 0.5836279988288879
INFO:lightwood-2822:Loss @ epoch 250: 0.5823927521705627
INFO:lightwood-2822:Loss @ epoch 251: 0.5813853740692139
INFO:lightwood-2822:Loss @ epoch 252: 0.5800496935844421
INFO:lightwood-2822:Loss @ epoch 253: 0.5767828822135925
INFO:lightwood-2822:Loss @ epoch 254: 0.5753887891769409
INFO:lightwood-2822:Loss @ epoch 255: 0.574046790599823
INFO:lightwood-2822:Loss @ epoch 256: 0.5728410482406616
INFO:lightwood-2822:Loss @ epoch 257: 0.5718717575073242
INFO:lightwood-2822:Loss @ epoch 258: 0.5705804228782654
INFO:lightwood-2822:Loss @ epoch 259: 0.567415177822113
INFO:lightwood-2822:Loss @ epoch 260: 0.5660616755485535
INFO:lightwood-2822:Loss @ epoch 261: 0.5647613406181335
INFO:lightwood-2822:Loss @ epoch 262: 0.5635983347892761
INFO:lightwood-2822:Loss @ epoch 263: 0.5626672506332397
INFO:lightwood-2822:Loss @ epoch 264: 0.5614020824432373
INFO:lightwood-2822:Loss @ epoch 265: 0.5582968592643738
INFO:lightwood-2822:Loss @ epoch 266: 0.5569714903831482
INFO:lightwood-2822:Loss @ epoch 267: 0.5556975603103638
INFO:lightwood-2822:Loss @ epoch 268: 0.5545614361763
INFO:lightwood-2822:Loss @ epoch 269: 0.5536586046218872
INFO:lightwood-2822:Loss @ epoch 270: 0.5524224638938904
INFO:lightwood-2822:Loss @ epoch 271: 0.5493913292884827
INFO:lightwood-2822:Loss @ epoch 272: 0.5480867028236389
INFO:lightwood-2822:Loss @ epoch 273: 0.5468233227729797
INFO:lightwood-2822:Loss @ epoch 274: 0.545693576335907
INFO:lightwood-2822:Loss @ epoch 275: 0.5447928309440613
INFO:lightwood-2822:Loss @ epoch 276: 0.5435579419136047
INFO:lightwood-2822:Loss @ epoch 277: 0.5405210256576538
INFO:lightwood-2822:Loss @ epoch 278: 0.5392288565635681
INFO:lightwood-2822:Loss @ epoch 279: 0.5379899740219116
INFO:lightwood-2822:Loss @ epoch 280: 0.5368918776512146
INFO:lightwood-2822:Loss @ epoch 281: 0.5360124707221985
INFO:lightwood-2822:Loss @ epoch 282: 0.5347910523414612
INFO:lightwood-2822:Loss @ epoch 283: 0.5317804217338562
INFO:lightwood-2822:Loss @ epoch 284: 0.5304878950119019
INFO:lightwood-2822:Loss @ epoch 285: 0.5292532444000244
INFO:lightwood-2822:Loss @ epoch 286: 0.5281679034233093
INFO:lightwood-2822:Loss @ epoch 287: 0.5273048281669617
INFO:lightwood-2822:Loss @ epoch 288: 0.526109516620636
INFO:lightwood-2822:Loss @ epoch 289: 0.5231714844703674
INFO:lightwood-2822:Loss @ epoch 290: 0.5219140648841858
INFO:lightwood-2822:Loss @ epoch 291: 0.5207134485244751
INFO:lightwood-2822:Loss @ epoch 292: 0.519664466381073
INFO:lightwood-2822:Loss @ epoch 293: 0.5188225507736206
INFO:lightwood-2822:Loss @ epoch 294: 0.5176380276679993
INFO:lightwood-2822:Loss @ epoch 295: 0.5147156119346619
INFO:lightwood-2822:Loss @ epoch 296: 0.5134611129760742
INFO:lightwood-2822:Loss @ epoch 297: 0.512263298034668
INFO:lightwood-2822:Loss @ epoch 298: 0.511226236820221
INFO:lightwood-2822:Loss @ epoch 299: 0.5103998184204102
INFO:lightwood-2822:Loss @ epoch 300: 0.5092394351959229
INFO:lightwood-2822:Loss @ epoch 301: 0.5063633322715759
INFO:lightwood-2822:Loss @ epoch 302: 0.5051231384277344
INFO:lightwood-2822:Loss @ epoch 303: 0.5039354562759399
INFO:lightwood-2822:Loss @ epoch 304: 0.502917468547821
INFO:lightwood-2822:Loss @ epoch 305: 0.5021116137504578
INFO:lightwood-2822:Loss @ epoch 306: 0.5009737014770508
INFO:lightwood-2822:Loss @ epoch 307: 0.49816152453422546
INFO:lightwood-2822:Loss @ epoch 308: 0.4969501793384552
INFO:lightwood-2822:Loss @ epoch 309: 0.495792955160141
INFO:lightwood-2822:Loss @ epoch 310: 0.4947965145111084
INFO:lightwood-2822:Loss @ epoch 311: 0.494009405374527
INFO:lightwood-2822:Loss @ epoch 312: 0.49289312958717346
INFO:lightwood-2822:Loss @ epoch 313: 0.49011099338531494
INFO:lightwood-2822:Loss @ epoch 314: 0.4889059364795685
INFO:lightwood-2822:Loss @ epoch 315: 0.48775631189346313
INFO:lightwood-2822:Loss @ epoch 316: 0.4867849051952362
INFO:lightwood-2822:Loss @ epoch 317: 0.4860127866268158
INFO:lightwood-2822:Loss @ epoch 318: 0.4849153459072113
INFO:lightwood-2822:Loss @ epoch 319: 0.482190877199173
INFO:lightwood-2822:Loss @ epoch 320: 0.4810042381286621
INFO:lightwood-2822:Loss @ epoch 321: 0.47987285256385803
INFO:lightwood-2822:Loss @ epoch 322: 0.4789106249809265
INFO:lightwood-2822:Loss @ epoch 323: 0.4781520664691925
INFO:lightwood-2822:Loss @ epoch 324: 0.4770645201206207
INFO:lightwood-2822:Loss @ epoch 325: 0.4743651747703552
INFO:lightwood-2822:Loss @ epoch 326: 0.4732038676738739
INFO:lightwood-2822:Loss @ epoch 327: 0.4720975458621979
INFO:lightwood-2822:Loss @ epoch 328: 0.47116589546203613
INFO:lightwood-2822:Loss @ epoch 329: 0.4704287350177765
INFO:lightwood-2822:Loss @ epoch 330: 0.46936970949172974
INFO:lightwood-2822:Loss @ epoch 331: 0.4667299687862396
INFO:lightwood-2822:Loss @ epoch 332: 0.4655860960483551
INFO:lightwood-2822:Loss @ epoch 333: 0.4644981026649475
INFO:lightwood-2822:Loss @ epoch 334: 0.4635782241821289
INFO:lightwood-2822:Loss @ epoch 335: 0.4628423750400543
INFO:lightwood-2822:Loss @ epoch 336: 0.46178358793258667
INFO:lightwood-2822:Loss @ epoch 337: 0.4591390788555145
INFO:lightwood-2822:Loss @ epoch 338: 0.45799314975738525
INFO:lightwood-2822:Loss @ epoch 339: 0.45690658688545227
INFO:lightwood-2822:Loss @ epoch 340: 0.455995112657547
INFO:lightwood-2822:Loss @ epoch 341: 0.45527079701423645
INFO:lightwood-2822:Loss @ epoch 342: 0.45423007011413574
INFO:lightwood-2822:Loss @ epoch 343: 0.4516132175922394
INFO:lightwood-2822:Loss @ epoch 344: 0.4504717290401459
INFO:lightwood-2822:Loss @ epoch 345: 0.44938918948173523
INFO:lightwood-2822:Loss @ epoch 346: 0.44849085807800293
INFO:lightwood-2822:Loss @ epoch 347: 0.44776931405067444
INFO:lightwood-2822:Loss @ epoch 348: 0.44672903418540955
INFO:lightwood-2822:Loss @ epoch 349: 0.4441067576408386
INFO:lightwood-2822:Loss @ epoch 350: 0.44297000765800476
INFO:lightwood-2822:Loss @ epoch 351: 0.44188418984413147
INFO:lightwood-2822:Loss @ epoch 352: 0.4409778416156769
INFO:lightwood-2822:Loss @ epoch 353: 0.4402410089969635
INFO:lightwood-2822:Loss @ epoch 354: 0.43918585777282715
INFO:lightwood-2822:Loss @ epoch 355: 0.43652641773223877
INFO:lightwood-2822:Loss @ epoch 356: 0.43538206815719604
INFO:lightwood-2822:Loss @ epoch 357: 0.43430009484291077
INFO:lightwood-2822:Loss @ epoch 358: 0.4333982467651367
INFO:lightwood-2822:Loss @ epoch 359: 0.4326588213443756
INFO:lightwood-2822:Loss @ epoch 360: 0.4316176772117615
INFO:lightwood-2822:Loss @ epoch 361: 0.4290023744106293
INFO:lightwood-2822:Loss @ epoch 362: 0.42786669731140137
INFO:lightwood-2822:Loss @ epoch 363: 0.426777184009552
INFO:lightwood-2822:Loss @ epoch 364: 0.42586755752563477
INFO:lightwood-2822:Loss @ epoch 365: 0.4251071810722351
INFO:lightwood-2822:Loss @ epoch 366: 0.4240370988845825
INFO:lightwood-2822:Loss @ epoch 367: 0.42131349444389343
INFO:lightwood-2822:Loss @ epoch 368: 0.42014428973197937
INFO:lightwood-2822:Loss @ epoch 369: 0.4190414547920227
INFO:lightwood-2822:Loss @ epoch 370: 0.4181329905986786
INFO:lightwood-2822:Loss @ epoch 371: 0.41738998889923096
INFO:lightwood-2822:Loss @ epoch 372: 0.41638195514678955
INFO:lightwood-2822:Loss @ epoch 373: 0.41390180587768555
INFO:lightwood-2822:Loss @ epoch 374: 0.412866473197937
INFO:lightwood-2822:Loss @ epoch 375: 0.41190284490585327
INFO:lightwood-2822:Loss @ epoch 376: 0.4111414849758148
INFO:lightwood-2822:Loss @ epoch 377: 0.41052889823913574
INFO:lightwood-2822:Loss @ epoch 378: 0.4096294939517975
INFO:lightwood-2822:Loss @ epoch 379: 0.407349169254303
INFO:lightwood-2822:Loss @ epoch 380: 0.4063820540904999
INFO:lightwood-2822:Loss @ epoch 381: 0.405462384223938
INFO:lightwood-2822:Loss @ epoch 382: 0.40475305914878845
INFO:lightwood-2822:Loss @ epoch 383: 0.40418320894241333
INFO:lightwood-2822:Loss @ epoch 384: 0.4033387005329132
INFO:lightwood-2822:Loss @ epoch 385: 0.4011896252632141
INFO:lightwood-2822:Loss @ epoch 386: 0.4002435803413391
INFO:lightwood-2822:Loss @ epoch 387: 0.39935779571533203
INFO:lightwood-2822:Loss @ epoch 388: 0.3986775577068329
INFO:lightwood-2822:Loss @ epoch 389: 0.3981267213821411
INFO:lightwood-2822:Loss @ epoch 390: 0.39729809761047363
INFO:lightwood-2822:Loss @ epoch 391: 0.39517998695373535
INFO:lightwood-2822:Loss @ epoch 392: 0.39425355195999146
INFO:lightwood-2822:Loss @ epoch 393: 0.3933902978897095
INFO:lightwood-2822:Loss @ epoch 394: 0.39273905754089355
INFO:lightwood-2822:Loss @ epoch 395: 0.39221012592315674
INFO:lightwood-2822:Loss @ epoch 396: 0.39140641689300537
INFO:lightwood-2822:Loss @ epoch 397: 0.389339804649353
INFO:lightwood-2822:Loss @ epoch 398: 0.38843241333961487
INFO:lightwood-2822:Loss @ epoch 399: 0.38758882880210876
INFO:lightwood-2822:Loss @ epoch 400: 0.3869537115097046
INFO:lightwood-2822:Loss @ epoch 401: 0.386430561542511
INFO:lightwood-2822:Loss @ epoch 402: 0.38564103841781616
INFO:lightwood-2822:Loss @ epoch 403: 0.38360342383384705
INFO:lightwood-2822:Loss @ epoch 404: 0.3827012777328491
INFO:lightwood-2822:Loss @ epoch 405: 0.3818664848804474
INFO:lightwood-2822:Loss @ epoch 406: 0.3812609910964966
INFO:lightwood-2822:Loss @ epoch 407: 0.3807579576969147
INFO:lightwood-2822:Loss @ epoch 408: 0.3799981474876404
INFO:lightwood-2822:Loss @ epoch 409: 0.3780278265476227
INFO:lightwood-2822:Loss @ epoch 410: 0.37715527415275574
INFO:lightwood-2822:Loss @ epoch 411: 0.3763349950313568
INFO:lightwood-2822:Loss @ epoch 412: 0.37574055790901184
INFO:lightwood-2822:Loss @ epoch 413: 0.37524667382240295
INFO:lightwood-2822:Loss @ epoch 414: 0.37449491024017334
INFO:lightwood-2822:Loss @ epoch 415: 0.3725510835647583
INFO:lightwood-2822:Loss @ epoch 416: 0.3717007040977478
INFO:lightwood-2822:Loss @ epoch 417: 0.37091028690338135
INFO:lightwood-2822:Loss @ epoch 418: 0.37033817172050476
INFO:lightwood-2822:Loss @ epoch 419: 0.3698543906211853
INFO:lightwood-2822:Loss @ epoch 420: 0.3691248297691345
INFO:lightwood-2822:Loss @ epoch 421: 0.3672463595867157
INFO:lightwood-2822:Loss @ epoch 422: 0.3664213716983795
INFO:lightwood-2822:Loss @ epoch 423: 0.3656456470489502
INFO:lightwood-2822:Loss @ epoch 424: 0.365101158618927
INFO:lightwood-2822:Loss @ epoch 425: 0.3646385371685028
INFO:lightwood-2822:Loss @ epoch 426: 0.363930344581604
INFO:lightwood-2822:Loss @ epoch 427: 0.3620816767215729
INFO:lightwood-2822:Loss @ epoch 428: 0.36128097772598267
INFO:lightwood-2822:Loss @ epoch 429: 0.3605450987815857
INFO:lightwood-2822:Loss @ epoch 430: 0.36003440618515015
INFO:lightwood-2822:Loss @ epoch 431: 0.3595956861972809
INFO:lightwood-2822:Loss @ epoch 432: 0.35892221331596375
INFO:lightwood-2822:Loss @ epoch 433: 0.35714542865753174
INFO:lightwood-2822:Loss @ epoch 434: 0.3563515245914459
INFO:lightwood-2822:Loss @ epoch 435: 0.3556281626224518
INFO:lightwood-2822:Loss @ epoch 436: 0.3551281988620758
INFO:lightwood-2822:Loss @ epoch 437: 0.3546939194202423
INFO:lightwood-2822:Loss @ epoch 438: 0.35402897000312805
INFO:lightwood-2822:Loss @ epoch 439: 0.3522771894931793
INFO:lightwood-2822:Loss @ epoch 440: 0.35150691866874695
INFO:lightwood-2822:Loss @ epoch 441: 0.3507969081401825
INFO:lightwood-2822:Loss @ epoch 442: 0.3503190279006958
INFO:lightwood-2822:Loss @ epoch 443: 0.34989500045776367
INFO:lightwood-2822:Loss @ epoch 444: 0.349252313375473
INFO:lightwood-2822:Loss @ epoch 445: 0.34754228591918945
INFO:lightwood-2822:Loss @ epoch 446: 0.3467896282672882
INFO:lightwood-2822:Loss @ epoch 447: 0.3460943400859833
INFO:lightwood-2822:Loss @ epoch 448: 0.34562933444976807
INFO:lightwood-2822:Loss @ epoch 449: 0.34521985054016113
INFO:lightwood-2822:Loss @ epoch 450: 0.3445984423160553
INFO:lightwood-2822:Loss @ epoch 451: 0.34293630719184875
INFO:lightwood-2822:Loss @ epoch 452: 0.3422030210494995
INFO:lightwood-2822:Loss @ epoch 453: 0.34152862429618835
INFO:lightwood-2822:Loss @ epoch 454: 0.34108397364616394
INFO:lightwood-2822:Loss @ epoch 455: 0.34068670868873596
INFO:lightwood-2822:Loss @ epoch 456: 0.340080201625824
INFO:lightwood-2822:Loss @ epoch 457: 0.3384317457675934
INFO:lightwood-2822:Loss @ epoch 458: 0.3376965820789337
INFO:lightwood-2822:Loss @ epoch 459: 0.33702588081359863
INFO:lightwood-2822:Loss @ epoch 460: 0.3365907669067383
INFO:lightwood-2822:Loss @ epoch 461: 0.33619338274002075
INFO:lightwood-2822:Loss @ epoch 462: 0.3356015384197235
INFO:lightwood-2822:Loss @ epoch 463: 0.33399897813796997
INFO:lightwood-2822:Loss @ epoch 464: 0.3332940638065338
INFO:lightwood-2822:Loss @ epoch 465: 0.33264991641044617
INFO:lightwood-2822:Loss @ epoch 466: 0.3322429656982422
INFO:lightwood-2822:Loss @ epoch 467: 0.3318733870983124
INFO:lightwood-2822:Loss @ epoch 468: 0.3313138484954834
INFO:lightwood-2822:Loss @ epoch 469: 0.3297865092754364
INFO:lightwood-2822:Loss @ epoch 470: 0.3290959894657135
INFO:lightwood-2822:Loss @ epoch 471: 0.3284758925437927
INFO:lightwood-2822:Loss @ epoch 472: 0.3280809223651886
INFO:lightwood-2822:Loss @ epoch 473: 0.327717661857605
INFO:lightwood-2822:Loss @ epoch 474: 0.32716163992881775
INFO:lightwood-2822:Loss @ epoch 475: 0.32564887404441833
INFO:lightwood-2822:Loss @ epoch 476: 0.32497289776802063
INFO:lightwood-2822:Loss @ epoch 477: 0.3243508040904999
INFO:lightwood-2822:Loss @ epoch 478: 0.32396966218948364
INFO:lightwood-2822:Loss @ epoch 479: 0.3235945999622345
INFO:lightwood-2822:Loss @ epoch 480: 0.3230462670326233
INFO:lightwood-2822:Loss @ epoch 481: 0.3215424120426178
INFO:lightwood-2822:Loss @ epoch 482: 0.3208779990673065
INFO:lightwood-2822:Loss @ epoch 483: 0.32027241587638855
INFO:lightwood-2822:Loss @ epoch 484: 0.31989675760269165
INFO:lightwood-2822:Loss @ epoch 485: 0.31951776146888733
INFO:lightwood-2822:Loss @ epoch 486: 0.3189714848995209
INFO:lightwood-2822:Loss @ epoch 487: 0.3174654245376587
INFO:lightwood-2822:Loss @ epoch 488: 0.3168082535266876
INFO:lightwood-2822:Loss @ epoch 489: 0.31619924306869507
INFO:lightwood-2822:Loss @ epoch 490: 0.3158278465270996
INFO:lightwood-2822:Loss @ epoch 491: 0.3154618740081787
INFO:lightwood-2822:Loss @ epoch 492: 0.31493353843688965
INFO:lightwood-2822:Loss @ epoch 493: 0.3134651482105255
INFO:lightwood-2822:Loss @ epoch 494: 0.3128322958946228
INFO:lightwood-2822:Loss @ epoch 495: 0.31225335597991943
INFO:lightwood-2822:Loss @ epoch 496: 0.3119007647037506
INFO:lightwood-2822:Loss @ epoch 497: 0.3115481436252594
INFO:lightwood-2822:Loss @ epoch 498: 0.31104305386543274
INFO:lightwood-2822:Loss @ epoch 499: 0.30964452028274536
INFO:lightwood-2822:Loss @ epoch 500: 0.30903780460357666
INFO:lightwood-2822:Loss @ epoch 501: 0.3084820806980133
INFO:lightwood-2822:Loss @ epoch 502: 0.30814746022224426
INFO:lightwood-2822:Loss @ epoch 503: 0.30780768394470215
INFO:lightwood-2822:Loss @ epoch 504: 0.3073209226131439
INFO:lightwood-2822:Loss @ epoch 505: 0.30594682693481445
INFO:lightwood-2822:Loss @ epoch 506: 0.3053426742553711
INFO:lightwood-2822:Loss @ epoch 507: 0.3047827184200287
INFO:lightwood-2822:Loss @ epoch 508: 0.3044542670249939
INFO:lightwood-2822:Loss @ epoch 509: 0.30410847067832947
INFO:lightwood-2822:Loss @ epoch 510: 0.3036235272884369
INFO:lightwood-2822:Loss @ epoch 511: 0.30225181579589844
INFO:lightwood-2822:Loss @ epoch 512: 0.30165547132492065
INFO:lightwood-2822:Loss @ epoch 513: 0.3011057674884796
INFO:lightwood-2822:Loss @ epoch 514: 0.30078256130218506
INFO:lightwood-2822:Loss @ epoch 515: 0.30042868852615356
INFO:lightwood-2822:Loss @ epoch 516: 0.29995155334472656
INFO:lightwood-2822:Loss @ epoch 517: 0.2985835075378418
INFO:lightwood-2822:Loss @ epoch 518: 0.29798927903175354
INFO:lightwood-2822:Loss @ epoch 519: 0.29743310809135437
INFO:lightwood-2822:Loss @ epoch 520: 0.2971104383468628
INFO:lightwood-2822:Loss @ epoch 521: 0.2967546880245209
INFO:lightwood-2822:Loss @ epoch 522: 0.29628410935401917
INFO:lightwood-2822:Loss @ epoch 523: 0.29493069648742676
INFO:lightwood-2822:Loss @ epoch 524: 0.29434067010879517
INFO:lightwood-2822:Loss @ epoch 525: 0.29379919171333313
INFO:lightwood-2822:Loss @ epoch 526: 0.2934878468513489
INFO:lightwood-2822:Loss @ epoch 527: 0.29313626885414124
INFO:lightwood-2822:Loss @ epoch 528: 0.2926904559135437
INFO:lightwood-2822:Loss @ epoch 529: 0.2913893759250641
INFO:lightwood-2822:Loss @ epoch 530: 0.2908231019973755
INFO:lightwood-2822:Loss @ epoch 531: 0.2903068959712982
INFO:lightwood-2822:Loss @ epoch 532: 0.2900163531303406
INFO:lightwood-2822:Loss @ epoch 533: 0.2896748483181
INFO:lightwood-2822:Loss @ epoch 534: 0.2892322242259979
INFO:lightwood-2822:Loss @ epoch 535: 0.28791794180870056
INFO:lightwood-2822:Loss @ epoch 536: 0.2873426377773285
INFO:lightwood-2822:Loss @ epoch 537: 0.2868235111236572
INFO:lightwood-2822:Loss @ epoch 538: 0.2865385115146637
INFO:lightwood-2822:Loss @ epoch 539: 0.2862080931663513
INFO:lightwood-2822:Loss @ epoch 540: 0.28579187393188477
INFO:lightwood-2822:Loss @ epoch 541: 0.2845536768436432
INFO:lightwood-2822:Loss @ epoch 542: 0.2840174436569214
INFO:lightwood-2822:Loss @ epoch 543: 0.2835312783718109
INFO:lightwood-2822:Loss @ epoch 544: 0.2832720875740051
INFO:lightwood-2822:Loss @ epoch 545: 0.282956063747406
INFO:lightwood-2822:Loss @ epoch 546: 0.2825598418712616
INFO:lightwood-2822:Loss @ epoch 547: 0.2813543975353241
INFO:lightwood-2822:Loss @ epoch 548: 0.2808196246623993
INFO:lightwood-2822:Loss @ epoch 549: 0.28034478425979614
INFO:lightwood-2822:Loss @ epoch 550: 0.2800978720188141
INFO:lightwood-2822:Loss @ epoch 551: 0.27977728843688965
INFO:lightwood-2822:Loss @ epoch 552: 0.2793889343738556
INFO:lightwood-2822:Loss @ epoch 553: 0.2781955897808075
INFO:lightwood-2822:Loss @ epoch 554: 0.2776784598827362
INFO:lightwood-2822:Loss @ epoch 555: 0.27720972895622253
INFO:lightwood-2822:Loss @ epoch 556: 0.27697333693504333
INFO:lightwood-2822:Loss @ epoch 557: 0.27666300535202026
INFO:lightwood-2822:Loss @ epoch 558: 0.2762896716594696
INFO:lightwood-2822:Loss @ epoch 559: 0.275119811296463
INFO:lightwood-2822:Loss @ epoch 560: 0.2745881676673889
INFO:lightwood-2822:Loss @ epoch 561: 0.2741168439388275
INFO:lightwood-2822:Loss @ epoch 562: 0.27388185262680054
INFO:lightwood-2822:Loss @ epoch 563: 0.2735639810562134
INFO:lightwood-2822:Loss @ epoch 564: 0.2731940746307373
INFO:lightwood-2822:Loss @ epoch 565: 0.27202731370925903
INFO:lightwood-2822:Loss @ epoch 566: 0.27150729298591614
INFO:lightwood-2822:Loss @ epoch 567: 0.271043986082077
INFO:lightwood-2822:Loss @ epoch 568: 0.27081188559532166
INFO:lightwood-2822:Loss @ epoch 569: 0.27049851417541504
INFO:lightwood-2822:Loss @ epoch 570: 0.27014273405075073
INFO:lightwood-2822:Loss @ epoch 571: 0.26900380849838257
INFO:lightwood-2822:Loss @ epoch 572: 0.2684919536113739
INFO:lightwood-2822:Loss @ epoch 573: 0.2680283188819885
INFO:lightwood-2822:Loss @ epoch 574: 0.26780539751052856
INFO:lightwood-2822:Loss @ epoch 575: 0.26748764514923096
INFO:lightwood-2822:Loss @ epoch 576: 0.2671354413032532
INFO:lightwood-2822:Loss @ epoch 577: 0.2660016119480133
INFO:lightwood-2822:Loss @ epoch 578: 0.26549026370048523
INFO:lightwood-2822:Loss @ epoch 579: 0.2650386393070221
INFO:lightwood-2822:Loss @ epoch 580: 0.26482194662094116
INFO:lightwood-2822:Loss @ epoch 581: 0.26450133323669434
INFO:lightwood-2822:Loss @ epoch 582: 0.2641564905643463
INFO:lightwood-2822:Loss @ epoch 583: 0.2630331516265869
INFO:lightwood-2822:Loss @ epoch 584: 0.2625267803668976
INFO:lightwood-2822:Loss @ epoch 585: 0.2620759606361389
INFO:lightwood-2822:Loss @ epoch 586: 0.261854350566864
INFO:lightwood-2822:Loss @ epoch 587: 0.2615257799625397
INFO:lightwood-2822:Loss @ epoch 588: 0.26118728518486023
INFO:lightwood-2822:Loss @ epoch 589: 0.2600722014904022
INFO:lightwood-2822:Loss @ epoch 590: 0.2595798671245575
INFO:lightwood-2822:Loss @ epoch 591: 0.25913041830062866
INFO:lightwood-2822:Loss @ epoch 592: 0.2589055299758911
INFO:lightwood-2822:Loss @ epoch 593: 0.25856778025627136
INFO:lightwood-2822:Loss @ epoch 594: 0.2582330107688904
INFO:lightwood-2822:Loss @ epoch 595: 0.2571284770965576
INFO:lightwood-2822:Loss @ epoch 596: 0.25662946701049805
INFO:lightwood-2822:Loss @ epoch 597: 0.2561897933483124
INFO:lightwood-2822:Loss @ epoch 598: 0.25598445534706116
INFO:lightwood-2822:Loss @ epoch 599: 0.25564634799957275
INFO:lightwood-2822:Loss @ epoch 600: 0.2553386390209198
INFO:lightwood-2822:Loss @ epoch 601: 0.2543119192123413
INFO:lightwood-2822:Loss @ epoch 602: 0.2538599967956543
INFO:lightwood-2822:Loss @ epoch 603: 0.25346454977989197
INFO:lightwood-2822:Loss @ epoch 604: 0.253299355506897
INFO:lightwood-2822:Loss @ epoch 605: 0.2529993951320648
INFO:lightwood-2822:Loss @ epoch 606: 0.25271809101104736
INFO:lightwood-2822:Loss @ epoch 607: 0.2517348527908325
INFO:lightwood-2822:Loss @ epoch 608: 0.25128936767578125
INFO:lightwood-2822:Loss @ epoch 609: 0.25089624524116516
INFO:lightwood-2822:Loss @ epoch 610: 0.2507348358631134
INFO:lightwood-2822:Loss @ epoch 611: 0.25042924284935
INFO:lightwood-2822:Loss @ epoch 612: 0.2501460015773773
INFO:lightwood-2822:Loss @ epoch 613: 0.24915650486946106
INFO:lightwood-2822:Loss @ epoch 614: 0.2487189769744873
INFO:lightwood-2822:Loss @ epoch 615: 0.248335063457489
INFO:lightwood-2822:Loss @ epoch 616: 0.24817804992198944
INFO:lightwood-2822:Loss @ epoch 617: 0.24787108600139618
INFO:lightwood-2822:Loss @ epoch 618: 0.24759645760059357
INFO:lightwood-2822:Loss @ epoch 619: 0.24661752581596375
INFO:lightwood-2822:Loss @ epoch 620: 0.24618574976921082
INFO:lightwood-2822:Loss @ epoch 621: 0.2457968294620514
INFO:lightwood-2822:Loss @ epoch 622: 0.24563506245613098
INFO:lightwood-2822:Loss @ epoch 623: 0.2453174889087677
INFO:lightwood-2822:Loss @ epoch 624: 0.24504433572292328
INFO:lightwood-2822:Loss @ epoch 625: 0.2440740168094635
INFO:lightwood-2822:Loss @ epoch 626: 0.24362798035144806
INFO:lightwood-2822:Loss @ epoch 627: 0.2432490587234497
INFO:lightwood-2822:Loss @ epoch 628: 0.24308602511882782
INFO:lightwood-2822:Loss @ epoch 629: 0.24276387691497803
INFO:lightwood-2822:Loss @ epoch 630: 0.2424861192703247
INFO:lightwood-2822:Loss @ epoch 631: 0.24150723218917847
INFO:lightwood-2822:Loss @ epoch 632: 0.24107399582862854
INFO:lightwood-2822:Loss @ epoch 633: 0.24068781733512878
INFO:lightwood-2822:Loss @ epoch 634: 0.24051880836486816
INFO:lightwood-2822:Loss @ epoch 635: 0.2401854693889618
INFO:lightwood-2822:Loss @ epoch 636: 0.2399105727672577
INFO:lightwood-2822:Loss @ epoch 637: 0.2389436662197113
INFO:lightwood-2822:Loss @ epoch 638: 0.2385157346725464
INFO:lightwood-2822:Loss @ epoch 639: 0.23814137279987335
INFO:lightwood-2822:Loss @ epoch 640: 0.23798717558383942
INFO:lightwood-2822:Loss @ epoch 641: 0.23766139149665833
INFO:lightwood-2822:Loss @ epoch 642: 0.2373955398797989
INFO:lightwood-2822:Loss @ epoch 643: 0.2364223152399063
INFO:lightwood-2822:Loss @ epoch 644: 0.23600433766841888
INFO:lightwood-2822:Loss @ epoch 645: 0.23563921451568604
INFO:lightwood-2822:Loss @ epoch 646: 0.23548835515975952
INFO:lightwood-2822:Loss @ epoch 647: 0.235157772898674
INFO:lightwood-2822:Loss @ epoch 648: 0.23489625751972198
INFO:lightwood-2822:Loss @ epoch 649: 0.233942449092865
INFO:lightwood-2822:Loss @ epoch 650: 0.23352423310279846
INFO:lightwood-2822:Loss @ epoch 651: 0.2331562340259552
INFO:lightwood-2822:Loss @ epoch 652: 0.23301081359386444
INFO:lightwood-2822:Loss @ epoch 653: 0.23267175257205963
INFO:lightwood-2822:Loss @ epoch 654: 0.23241490125656128
INFO:lightwood-2822:Loss @ epoch 655: 0.23146721720695496
INFO:lightwood-2822:Loss @ epoch 656: 0.231054425239563
INFO:lightwood-2822:Loss @ epoch 657: 0.23069924116134644
INFO:lightwood-2822:Loss @ epoch 658: 0.2305404394865036
INFO:lightwood-2822:Loss @ epoch 659: 0.23020099103450775
INFO:lightwood-2822:Loss @ epoch 660: 0.22994983196258545
INFO:lightwood-2822:Loss @ epoch 661: 0.2290249466896057
INFO:lightwood-2822:Loss @ epoch 662: 0.22860360145568848
INFO:lightwood-2822:Loss @ epoch 663: 0.22826360166072845
INFO:lightwood-2822:Loss @ epoch 664: 0.22813338041305542
INFO:lightwood-2822:Loss @ epoch 665: 0.22779394686222076
INFO:lightwood-2822:Loss @ epoch 666: 0.2275463193655014
INFO:lightwood-2822:Loss @ epoch 667: 0.2266179919242859
INFO:lightwood-2822:Loss @ epoch 668: 0.22621648013591766
INFO:lightwood-2822:Loss @ epoch 669: 0.2258809208869934
INFO:lightwood-2822:Loss @ epoch 670: 0.22574372589588165
INFO:lightwood-2822:Loss @ epoch 671: 0.22540859878063202
INFO:lightwood-2822:Loss @ epoch 672: 0.22516551613807678
INFO:lightwood-2822:Loss @ epoch 673: 0.22424499690532684
INFO:lightwood-2822:Loss @ epoch 674: 0.2238486409187317
INFO:lightwood-2822:Loss @ epoch 675: 0.223518967628479
INFO:lightwood-2822:Loss @ epoch 676: 0.22337649762630463
INFO:lightwood-2822:Loss @ epoch 677: 0.22302983701229095
INFO:lightwood-2822:Loss @ epoch 678: 0.2227952778339386
INFO:lightwood-2822:Loss @ epoch 679: 0.22189918160438538
INFO:lightwood-2822:Loss @ epoch 680: 0.2214922159910202
INFO:lightwood-2822:Loss @ epoch 681: 0.2211611568927765
INFO:lightwood-2822:Loss @ epoch 682: 0.22102764248847961
INFO:lightwood-2822:Loss @ epoch 683: 0.22068741917610168
INFO:lightwood-2822:Loss @ epoch 684: 0.22045622766017914
INFO:lightwood-2822:Loss @ epoch 685: 0.21956941485404968
INFO:lightwood-2822:Loss @ epoch 686: 0.21916718780994415
INFO:lightwood-2822:Loss @ epoch 687: 0.21884170174598694
INFO:lightwood-2822:Loss @ epoch 688: 0.2187020629644394
INFO:lightwood-2822:Loss @ epoch 689: 0.21835395693778992
INFO:lightwood-2822:Loss @ epoch 690: 0.218128502368927
INFO:lightwood-2822:Loss @ epoch 691: 0.21726460754871368
INFO:lightwood-2822:Loss @ epoch 692: 0.21687211096286774
INFO:lightwood-2822:Loss @ epoch 693: 0.21654966473579407
INFO:lightwood-2822:Loss @ epoch 694: 0.21642103791236877
INFO:lightwood-2822:Loss @ epoch 695: 0.21607361733913422
INFO:lightwood-2822:Loss @ epoch 696: 0.21584638953208923
INFO:lightwood-2822:Loss @ epoch 697: 0.21497054398059845
INFO:lightwood-2822:Loss @ epoch 698: 0.21457690000534058
INFO:lightwood-2822:Loss @ epoch 699: 0.21425464749336243
INFO:lightwood-2822:Loss @ epoch 700: 0.21412666141986847
INFO:lightwood-2822:Loss @ epoch 701: 0.21377936005592346
INFO:lightwood-2822:Loss @ epoch 702: 0.2135673463344574
INFO:lightwood-2822:Loss @ epoch 703: 0.2127339392900467
INFO:lightwood-2822:Loss @ epoch 704: 0.2123561054468155
INFO:lightwood-2822:Loss @ epoch 705: 0.21205425262451172
INFO:lightwood-2822:Loss @ epoch 706: 0.21193821728229523
INFO:lightwood-2822:Loss @ epoch 707: 0.2115948647260666
INFO:lightwood-2822:Loss @ epoch 708: 0.21139240264892578
INFO:lightwood-2822:Loss @ epoch 709: 0.21057893335819244
INFO:lightwood-2822:Loss @ epoch 710: 0.21019414067268372
INFO:lightwood-2822:Loss @ epoch 711: 0.20988672971725464
INFO:lightwood-2822:Loss @ epoch 712: 0.20976302027702332
INFO:lightwood-2822:Loss @ epoch 713: 0.20940978825092316
INFO:lightwood-2822:Loss @ epoch 714: 0.2091984748840332
INFO:lightwood-2822:Loss @ epoch 715: 0.20836374163627625
INFO:lightwood-2822:Loss @ epoch 716: 0.2079932540655136
INFO:lightwood-2822:Loss @ epoch 717: 0.20770114660263062
INFO:lightwood-2822:Loss @ epoch 718: 0.2075936496257782
INFO:lightwood-2822:Loss @ epoch 719: 0.20724354684352875
INFO:lightwood-2822:Loss @ epoch 720: 0.20703889429569244
INFO:lightwood-2822:Loss @ epoch 721: 0.2062043845653534
INFO:lightwood-2822:Loss @ epoch 722: 0.20582519471645355
INFO:lightwood-2822:Loss @ epoch 723: 0.20553486049175262
INFO:lightwood-2822:Loss @ epoch 724: 0.20542825758457184
INFO:lightwood-2822:Loss @ epoch 725: 0.20507138967514038
INFO:lightwood-2822:Loss @ epoch 726: 0.20487387478351593
INFO:lightwood-2822:Loss @ epoch 727: 0.20405805110931396
INFO:lightwood-2822:Loss @ epoch 728: 0.20368054509162903
INFO:lightwood-2822:Loss @ epoch 729: 0.20338758826255798
INFO:lightwood-2822:Loss @ epoch 730: 0.2032729685306549
INFO:lightwood-2822:Loss @ epoch 731: 0.20290538668632507
INFO:lightwood-2822:Loss @ epoch 732: 0.20270086824893951
INFO:lightwood-2822:Loss @ epoch 733: 0.2018848955631256
INFO:lightwood-2822:Loss @ epoch 734: 0.20152272284030914
INFO:lightwood-2822:Loss @ epoch 735: 0.20122411847114563
INFO:lightwood-2822:Loss @ epoch 736: 0.2011106163263321
INFO:lightwood-2822:Loss @ epoch 737: 0.20074225962162018
INFO:lightwood-2822:Loss @ epoch 738: 0.20054370164871216
INFO:lightwood-2822:Loss @ epoch 739: 0.19974417984485626
INFO:lightwood-2822:Loss @ epoch 740: 0.19938015937805176
INFO:lightwood-2822:Loss @ epoch 741: 0.19909901916980743
INFO:lightwood-2822:Loss @ epoch 742: 0.19898763298988342
INFO:lightwood-2822:Loss @ epoch 743: 0.19862200319766998
INFO:lightwood-2822:Loss @ epoch 744: 0.19843274354934692
INFO:lightwood-2822:Loss @ epoch 745: 0.1976475566625595
INFO:lightwood-2822:Loss @ epoch 746: 0.19726091623306274
INFO:lightwood-2822:Loss @ epoch 747: 0.19696924090385437
INFO:lightwood-2822:Loss @ epoch 748: 0.196858212351799
INFO:lightwood-2822:Loss @ epoch 749: 0.1964864730834961
INFO:lightwood-2822:Loss @ epoch 750: 0.196295827627182
INFO:lightwood-2822:Loss @ epoch 751: 0.1955578625202179
INFO:lightwood-2822:Loss @ epoch 752: 0.1952170878648758
INFO:lightwood-2822:Loss @ epoch 753: 0.1949596107006073
INFO:lightwood-2822:Loss @ epoch 754: 0.19487544894218445
INFO:lightwood-2822:Loss @ epoch 755: 0.19453617930412292
INFO:lightwood-2822:Loss @ epoch 756: 0.19438514113426208
INFO:lightwood-2822:Loss @ epoch 757: 0.19369599223136902
INFO:lightwood-2822:Loss @ epoch 758: 0.19336649775505066
INFO:lightwood-2822:Loss @ epoch 759: 0.19311009347438812
INFO:lightwood-2822:Loss @ epoch 760: 0.19302169978618622
INFO:lightwood-2822:Loss @ epoch 761: 0.19268205761909485
INFO:lightwood-2822:Loss @ epoch 762: 0.19253277778625488
INFO:lightwood-2822:Loss @ epoch 763: 0.19185367226600647
INFO:lightwood-2822:Loss @ epoch 764: 0.19152258336544037
INFO:lightwood-2822:Loss @ epoch 765: 0.19129197299480438
INFO:lightwood-2822:Loss @ epoch 766: 0.1912132054567337
INFO:lightwood-2822:Loss @ epoch 767: 0.19086988270282745
INFO:lightwood-2822:Loss @ epoch 768: 0.19071801006793976
INFO:lightwood-2822:Loss @ epoch 769: 0.19003665447235107
INFO:lightwood-2822:Loss @ epoch 770: 0.1897275596857071
INFO:lightwood-2822:Loss @ epoch 771: 0.1894957721233368
INFO:lightwood-2822:Loss @ epoch 772: 0.1894221305847168
INFO:lightwood-2822:Loss @ epoch 773: 0.1890747994184494
INFO:lightwood-2822:Loss @ epoch 774: 0.18893224000930786
INFO:lightwood-2822:Loss @ epoch 775: 0.18826298415660858
INFO:lightwood-2822:Loss @ epoch 776: 0.18794694542884827
INFO:lightwood-2822:Loss @ epoch 777: 0.18771636486053467
INFO:lightwood-2822:Loss @ epoch 778: 0.18763215839862823
INFO:lightwood-2822:Loss @ epoch 779: 0.1872880905866623
INFO:lightwood-2822:Loss @ epoch 780: 0.18715108931064606
INFO:lightwood-2822:Loss @ epoch 781: 0.18649297952651978
INFO:lightwood-2822:Loss @ epoch 782: 0.18618376553058624
INFO:lightwood-2822:Loss @ epoch 783: 0.1859513819217682
INFO:lightwood-2822:Loss @ epoch 784: 0.18587908148765564
INFO:lightwood-2822:Loss @ epoch 785: 0.18553417921066284
INFO:lightwood-2822:Loss @ epoch 786: 0.1853959709405899
INFO:lightwood-2822:Loss @ epoch 787: 0.18474453687667847
INFO:lightwood-2822:Loss @ epoch 788: 0.1844433844089508
INFO:lightwood-2822:Loss @ epoch 789: 0.18425185978412628
INFO:lightwood-2822:Loss @ epoch 790: 0.18418721854686737
INFO:lightwood-2822:Loss @ epoch 791: 0.18384194374084473
INFO:lightwood-2822:Loss @ epoch 792: 0.18370750546455383
INFO:lightwood-2822:Loss @ epoch 793: 0.18307533860206604
INFO:lightwood-2822:Loss @ epoch 794: 0.18278977274894714
INFO:lightwood-2822:Loss @ epoch 795: 0.18258808553218842
INFO:lightwood-2822:Loss @ epoch 796: 0.18252550065517426
INFO:lightwood-2822:Loss @ epoch 797: 0.18218253552913666
INFO:lightwood-2822:Loss @ epoch 798: 0.18205450475215912
INFO:lightwood-2822:Loss @ epoch 799: 0.1814316362142563
INFO:lightwood-2822:Loss @ epoch 800: 0.18114912509918213
INFO:lightwood-2822:Loss @ epoch 801: 0.18096043169498444
INFO:lightwood-2822:Loss @ epoch 802: 0.1808912754058838
INFO:lightwood-2822:Loss @ epoch 803: 0.1805393546819687
INFO:lightwood-2822:Loss @ epoch 804: 0.18040260672569275
INFO:lightwood-2822:Loss @ epoch 805: 0.17976699769496918
INFO:lightwood-2822:Loss @ epoch 806: 0.1794717162847519
INFO:lightwood-2822:Loss @ epoch 807: 0.17926697432994843
INFO:lightwood-2822:Loss @ epoch 808: 0.17920295894145966
INFO:lightwood-2822:Loss @ epoch 809: 0.17884348332881927
INFO:lightwood-2822:Loss @ epoch 810: 0.178704634308815
INFO:lightwood-2822:Loss @ epoch 811: 0.17806199193000793
INFO:lightwood-2822:Loss @ epoch 812: 0.1777760088443756
INFO:lightwood-2822:Loss @ epoch 813: 0.17757587134838104
INFO:lightwood-2822:Loss @ epoch 814: 0.17750157415866852
INFO:lightwood-2822:Loss @ epoch 815: 0.17714524269104004
INFO:lightwood-2822:Loss @ epoch 816: 0.1770130693912506
INFO:lightwood-2822:Loss @ epoch 817: 0.17639745771884918
INFO:lightwood-2822:Loss @ epoch 818: 0.1760970950126648
INFO:lightwood-2822:Loss @ epoch 819: 0.17590491473674774
INFO:lightwood-2822:Loss @ epoch 820: 0.17583739757537842
INFO:lightwood-2822:Loss @ epoch 821: 0.17547473311424255
INFO:lightwood-2822:Loss @ epoch 822: 0.17534245550632477
INFO:lightwood-2822:Loss @ epoch 823: 0.17471250891685486
INFO:lightwood-2822:Loss @ epoch 824: 0.17441849410533905
INFO:lightwood-2822:Loss @ epoch 825: 0.17423683404922485
INFO:lightwood-2822:Loss @ epoch 826: 0.17417629063129425
INFO:lightwood-2822:Loss @ epoch 827: 0.17381568253040314
INFO:lightwood-2822:Loss @ epoch 828: 0.1736784726381302
INFO:lightwood-2822:Loss @ epoch 829: 0.1730453073978424
INFO:lightwood-2822:Loss @ epoch 830: 0.1727694422006607
INFO:lightwood-2822:Loss @ epoch 831: 0.1725722849369049
INFO:lightwood-2822:Loss @ epoch 832: 0.17249977588653564
INFO:lightwood-2822:Loss @ epoch 833: 0.17213518917560577
INFO:lightwood-2822:Loss @ epoch 834: 0.17200462520122528
INFO:lightwood-2822:Loss @ epoch 835: 0.17139704525470734
INFO:lightwood-2822:Loss @ epoch 836: 0.17111653089523315
INFO:lightwood-2822:Loss @ epoch 837: 0.17092281579971313
INFO:lightwood-2822:Loss @ epoch 838: 0.17085804045200348
INFO:lightwood-2822:Loss @ epoch 839: 0.1704958975315094
INFO:lightwood-2822:Loss @ epoch 840: 0.1703719049692154
INFO:lightwood-2822:Loss @ epoch 841: 0.16979235410690308
INFO:lightwood-2822:Loss @ epoch 842: 0.16953155398368835
INFO:lightwood-2822:Loss @ epoch 843: 0.16936221718788147
INFO:lightwood-2822:Loss @ epoch 844: 0.16931013762950897
INFO:lightwood-2822:Loss @ epoch 845: 0.16895925998687744
INFO:lightwood-2822:Loss @ epoch 846: 0.16885320842266083
INFO:lightwood-2822:Loss @ epoch 847: 0.16830115020275116
INFO:lightwood-2822:Loss @ epoch 848: 0.1680324524641037
INFO:lightwood-2822:Loss @ epoch 849: 0.16786083579063416
INFO:lightwood-2822:Loss @ epoch 850: 0.16780298948287964
INFO:lightwood-2822:Loss @ epoch 851: 0.16744723916053772
INFO:lightwood-2822:Loss @ epoch 852: 0.1673429161310196
INFO:lightwood-2822:Loss @ epoch 853: 0.1668015569448471
INFO:lightwood-2822:Loss @ epoch 854: 0.16653653979301453
INFO:lightwood-2822:Loss @ epoch 855: 0.16636568307876587
INFO:lightwood-2822:Loss @ epoch 856: 0.16631269454956055
INFO:lightwood-2822:Loss @ epoch 857: 0.16595155000686646
INFO:lightwood-2822:Loss @ epoch 858: 0.16584332287311554
INFO:lightwood-2822:Loss @ epoch 859: 0.16530001163482666
INFO:lightwood-2822:Loss @ epoch 860: 0.16503342986106873
INFO:lightwood-2822:Loss @ epoch 861: 0.16487862169742584
INFO:lightwood-2822:Loss @ epoch 862: 0.16483160853385925
INFO:lightwood-2822:Loss @ epoch 863: 0.16447648406028748
INFO:lightwood-2822:Loss @ epoch 864: 0.16437315940856934
INFO:lightwood-2822:Loss @ epoch 865: 0.16383177042007446
INFO:lightwood-2822:Loss @ epoch 866: 0.16356077790260315
INFO:lightwood-2822:Loss @ epoch 867: 0.16340747475624084
INFO:lightwood-2822:Loss @ epoch 868: 0.16335061192512512
INFO:lightwood-2822:Loss @ epoch 869: 0.1629944145679474
INFO:lightwood-2822:Loss @ epoch 870: 0.16289140284061432
INFO:lightwood-2822:Loss @ epoch 871: 0.16236847639083862
INFO:lightwood-2822:Loss @ epoch 872: 0.16211192309856415
INFO:lightwood-2822:Loss @ epoch 873: 0.16196134686470032
INFO:lightwood-2822:Loss @ epoch 874: 0.1619141399860382
INFO:lightwood-2822:Loss @ epoch 875: 0.16154593229293823
INFO:lightwood-2822:Loss @ epoch 876: 0.16144198179244995
INFO:lightwood-2822:Loss @ epoch 877: 0.16090774536132812
INFO:lightwood-2822:Loss @ epoch 878: 0.16064877808094025
INFO:lightwood-2822:Loss @ epoch 879: 0.16049553453922272
INFO:lightwood-2822:Loss @ epoch 880: 0.1604498028755188
INFO:lightwood-2822:Loss @ epoch 881: 0.16009260714054108
INFO:lightwood-2822:Loss @ epoch 882: 0.1599918156862259
INFO:lightwood-2822:Loss @ epoch 883: 0.15948525071144104
INFO:lightwood-2822:Loss @ epoch 884: 0.15921904146671295
INFO:lightwood-2822:Loss @ epoch 885: 0.15908163785934448
INFO:lightwood-2822:Loss @ epoch 886: 0.1590365469455719
INFO:lightwood-2822:Loss @ epoch 887: 0.15865720808506012
INFO:lightwood-2822:Loss @ epoch 888: 0.15855243802070618
INFO:lightwood-2822:Loss @ epoch 889: 0.15805572271347046
INFO:lightwood-2822:Loss @ epoch 890: 0.15781906247138977
INFO:lightwood-2822:Loss @ epoch 891: 0.15768083930015564
INFO:lightwood-2822:Loss @ epoch 892: 0.1576356738805771
INFO:lightwood-2822:Loss @ epoch 893: 0.1572713404893875
INFO:lightwood-2822:Loss @ epoch 894: 0.15717534720897675
INFO:lightwood-2822:Loss @ epoch 895: 0.15671366453170776
INFO:lightwood-2822:Loss @ epoch 896: 0.15647399425506592
INFO:lightwood-2822:Loss @ epoch 897: 0.1563211977481842
INFO:lightwood-2822:Loss @ epoch 898: 0.1562960147857666
INFO:lightwood-2822:Loss @ epoch 899: 0.15593737363815308
INFO:lightwood-2822:Loss @ epoch 900: 0.15584696829319
INFO:lightwood-2822:Loss @ epoch 901: 0.15537278354167938
INFO:lightwood-2822:Loss @ epoch 902: 0.15514235198497772
INFO:lightwood-2822:Loss @ epoch 903: 0.15501244366168976
INFO:lightwood-2822:Loss @ epoch 904: 0.15495038032531738
INFO:lightwood-2822:Loss @ epoch 905: 0.1545928716659546
INFO:lightwood-2822:Loss @ epoch 906: 0.15449456870555878
INFO:lightwood-2822:Loss @ epoch 907: 0.15401402115821838
INFO:lightwood-2822:Loss @ epoch 908: 0.15377826988697052
INFO:lightwood-2822:Loss @ epoch 909: 0.15366068482398987
INFO:lightwood-2822:Loss @ epoch 910: 0.15363460779190063
INFO:lightwood-2822:Loss @ epoch 911: 0.15328511595726013
INFO:lightwood-2822:Loss @ epoch 912: 0.15319031476974487
INFO:lightwood-2822:Loss @ epoch 913: 0.15270718932151794
INFO:lightwood-2822:Loss @ epoch 914: 0.15249189734458923
INFO:lightwood-2822:Loss @ epoch 915: 0.15237651765346527
INFO:lightwood-2822:Loss @ epoch 916: 0.15235473215579987
INFO:lightwood-2822:Loss @ epoch 917: 0.15200546383857727
INFO:lightwood-2822:Loss @ epoch 918: 0.15191559493541718
INFO:lightwood-2822:Loss @ epoch 919: 0.15145818889141083
INFO:lightwood-2822:Loss @ epoch 920: 0.15121348202228546
INFO:lightwood-2822:Loss @ epoch 921: 0.15110373497009277
INFO:lightwood-2822:Loss @ epoch 922: 0.15108099579811096
INFO:lightwood-2822:Loss @ epoch 923: 0.15072421729564667
INFO:lightwood-2822:Loss @ epoch 924: 0.1506342589855194
INFO:lightwood-2822:Loss @ epoch 925: 0.1501930058002472
INFO:lightwood-2822:Loss @ epoch 926: 0.14998698234558105
INFO:lightwood-2822:Loss @ epoch 927: 0.14987143874168396
INFO:lightwood-2822:Loss @ epoch 928: 0.14983835816383362
INFO:lightwood-2822:Loss @ epoch 929: 0.14947612583637238
INFO:lightwood-2822:Loss @ epoch 930: 0.14938704669475555
INFO:lightwood-2822:Loss @ epoch 931: 0.14895370602607727
INFO:lightwood-2822:Loss @ epoch 932: 0.14872637391090393
INFO:lightwood-2822:Loss @ epoch 933: 0.14861835539340973
INFO:lightwood-2822:Loss @ epoch 934: 0.14858628809452057
INFO:lightwood-2822:Loss @ epoch 935: 0.14823506772518158
INFO:lightwood-2822:Loss @ epoch 936: 0.14814943075180054
INFO:lightwood-2822:Loss @ epoch 937: 0.1477162390947342
INFO:lightwood-2822:Loss @ epoch 938: 0.14747311174869537
INFO:lightwood-2822:Loss @ epoch 939: 0.14736929535865784
INFO:lightwood-2822:Loss @ epoch 940: 0.14734449982643127
INFO:lightwood-2822:Loss @ epoch 941: 0.1469842940568924
INFO:lightwood-2822:Loss @ epoch 942: 0.14689628779888153
INFO:lightwood-2822:Loss @ epoch 943: 0.14647449553012848
INFO:lightwood-2822:Loss @ epoch 944: 0.14625394344329834
INFO:lightwood-2822:Loss @ epoch 945: 0.14617042243480682
INFO:lightwood-2822:Loss @ epoch 946: 0.1461440920829773
INFO:lightwood-2822:Loss @ epoch 947: 0.14579246938228607
INFO:lightwood-2822:Loss @ epoch 948: 0.14571082592010498
INFO:lightwood-2822:Loss @ epoch 949: 0.14528149366378784
INFO:lightwood-2822:Loss @ epoch 950: 0.1450585424900055
INFO:lightwood-2822:Loss @ epoch 951: 0.14496178925037384
INFO:lightwood-2822:Loss @ epoch 952: 0.14494746923446655
INFO:lightwood-2822:Loss @ epoch 953: 0.1445988416671753
INFO:lightwood-2822:Loss @ epoch 954: 0.14451491832733154
INFO:lightwood-2822:Loss @ epoch 955: 0.14409570395946503
INFO:lightwood-2822:Loss @ epoch 956: 0.14389833807945251
INFO:lightwood-2822:Loss @ epoch 957: 0.14382441341876984
INFO:lightwood-2822:Loss @ epoch 958: 0.14378409087657928
INFO:lightwood-2822:Loss @ epoch 959: 0.14342404901981354
INFO:lightwood-2822:Loss @ epoch 960: 0.14334389567375183
INFO:lightwood-2822:Loss @ epoch 961: 0.14294986426830292
INFO:lightwood-2822:Loss @ epoch 962: 0.1427377611398697
INFO:lightwood-2822:Loss @ epoch 963: 0.1426527500152588
INFO:lightwood-2822:Loss @ epoch 964: 0.14263136684894562
INFO:lightwood-2822:Loss @ epoch 965: 0.14226578176021576
INFO:lightwood-2822:Loss @ epoch 966: 0.14216598868370056
INFO:lightwood-2822:Loss @ epoch 967: 0.1417752504348755
INFO:lightwood-2822:Loss @ epoch 968: 0.1415865272283554
INFO:lightwood-2822:Loss @ epoch 969: 0.14149995148181915
INFO:lightwood-2822:Loss @ epoch 970: 0.1414736807346344
INFO:lightwood-2822:Loss @ epoch 971: 0.14111755788326263
INFO:lightwood-2822:Loss @ epoch 972: 0.14103908836841583
INFO:lightwood-2822:Loss @ epoch 973: 0.14067767560482025
INFO:lightwood-2822:Loss @ epoch 974: 0.14044982194900513
INFO:lightwood-2822:Loss @ epoch 975: 0.14036211371421814
INFO:lightwood-2822:Loss @ epoch 976: 0.14035210013389587
INFO:lightwood-2822:Loss @ epoch 977: 0.1399938017129898
INFO:lightwood-2822:Loss @ epoch 978: 0.1399068981409073
INFO:lightwood-2822:Loss @ epoch 979: 0.13952630758285522
INFO:lightwood-2822:Loss @ epoch 980: 0.1393301784992218
INFO:lightwood-2822:Loss @ epoch 981: 0.13923345506191254
INFO:lightwood-2822:Loss @ epoch 982: 0.1392124444246292
INFO:lightwood-2822:Loss @ epoch 983: 0.13886496424674988
INFO:lightwood-2822:Loss @ epoch 984: 0.1387902796268463
INFO:lightwood-2822:Loss @ epoch 985: 0.13843581080436707
INFO:lightwood-2822:Loss @ epoch 986: 0.13823610544204712
INFO:lightwood-2822:Loss @ epoch 987: 0.1381685584783554
INFO:lightwood-2822:Loss @ epoch 988: 0.1381262242794037
INFO:lightwood-2822:Loss @ epoch 989: 0.1377667784690857
INFO:lightwood-2822:Loss @ epoch 990: 0.13769327104091644
INFO:lightwood-2822:Loss @ epoch 991: 0.137346088886261
INFO:lightwood-2822:Loss @ epoch 992: 0.13713853061199188
INFO:lightwood-2822:Loss @ epoch 993: 0.13708150386810303
INFO:lightwood-2822:Loss @ epoch 994: 0.13705728948116302
INFO:lightwood-2822:Loss @ epoch 995: 0.13670390844345093
INFO:lightwood-2822:Loss @ epoch 996: 0.13662710785865784
INFO:lightwood-2822:Loss @ epoch 997: 0.1362839788198471
INFO:lightwood-2822:Loss @ epoch 998: 0.13610363006591797
INFO:lightwood-2822:Loss @ epoch 999: 0.13602496683597565
INFO:lightwood-2822:Loss @ epoch 1000: 0.13599584996700287
INFO:lightwood-2822:Loss @ epoch 1001: 0.13565266132354736
INFO:lightwood-2822:Loss @ epoch 1002: 0.1355796605348587
INFO:lightwood-2822:Loss @ epoch 1003: 0.13525879383087158
INFO:lightwood-2822:Loss @ epoch 1004: 0.13506895303726196
INFO:lightwood-2822:Loss @ epoch 1005: 0.13499005138874054
INFO:lightwood-2822:Loss @ epoch 1006: 0.13497398793697357
INFO:lightwood-2822:Loss @ epoch 1007: 0.13461682200431824
INFO:lightwood-2822:Loss @ epoch 1008: 0.1345367133617401
INFO:lightwood-2822:Loss @ epoch 1009: 0.13421472907066345
INFO:lightwood-2822:Loss @ epoch 1010: 0.1340406835079193
INFO:lightwood-2822:Loss @ epoch 1011: 0.13396066427230835
INFO:lightwood-2822:Loss @ epoch 1012: 0.13394540548324585
INFO:lightwood-2822:Loss @ epoch 1013: 0.133603036403656
INFO:lightwood-2822:Loss @ epoch 1014: 0.13353194296360016
INFO:lightwood-2822:Loss @ epoch 1015: 0.13320596516132355
INFO:lightwood-2822:Loss @ epoch 1016: 0.1330314576625824
INFO:lightwood-2822:Loss @ epoch 1017: 0.13297443091869354
INFO:lightwood-2822:Loss @ epoch 1018: 0.13295991718769073
INFO:lightwood-2822:Loss @ epoch 1019: 0.13259781897068024
INFO:lightwood-2822:Loss @ epoch 1020: 0.13251620531082153
INFO:lightwood-2822:Loss @ epoch 1021: 0.1322004646062851
INFO:lightwood-2822:Loss @ epoch 1022: 0.1320314109325409
INFO:lightwood-2822:Loss @ epoch 1023: 0.13197889924049377
INFO:lightwood-2822:Loss @ epoch 1024: 0.13195917010307312
INFO:lightwood-2822:Loss @ epoch 1025: 0.13160918653011322
INFO:lightwood-2822:Loss @ epoch 1026: 0.13154202699661255
INFO:lightwood-2822:Loss @ epoch 1027: 0.1312246471643448
INFO:lightwood-2822:Loss @ epoch 1028: 0.13103802502155304
INFO:lightwood-2822:Loss @ epoch 1029: 0.1309911459684372
INFO:lightwood-2822:Loss @ epoch 1030: 0.1309920698404312
INFO:lightwood-2822:Loss @ epoch 1031: 0.1306457221508026
INFO:lightwood-2822:Loss @ epoch 1032: 0.13056819140911102
INFO:lightwood-2822:Loss @ epoch 1033: 0.1302425116300583
INFO:lightwood-2822:Loss @ epoch 1034: 0.1300876885652542
INFO:lightwood-2822:Loss @ epoch 1035: 0.1300475150346756
INFO:lightwood-2822:Loss @ epoch 1036: 0.13001790642738342
INFO:lightwood-2822:Loss @ epoch 1037: 0.1296614706516266
INFO:lightwood-2822:Loss @ epoch 1038: 0.12959393858909607
INFO:lightwood-2822:Loss @ epoch 1039: 0.12931062281131744
INFO:lightwood-2822:Loss @ epoch 1040: 0.1291191130876541
INFO:lightwood-2822:Loss @ epoch 1041: 0.12908804416656494
INFO:lightwood-2822:Loss @ epoch 1042: 0.12908202409744263
INFO:lightwood-2822:Loss @ epoch 1043: 0.1287422776222229
INFO:lightwood-2822:Loss @ epoch 1044: 0.12866781651973724
INFO:lightwood-2822:Loss @ epoch 1045: 0.12837623059749603
INFO:lightwood-2822:Loss @ epoch 1046: 0.1282041072845459
INFO:lightwood-2822:Loss @ epoch 1047: 0.12815770506858826
INFO:lightwood-2822:Loss @ epoch 1048: 0.12815126776695251
INFO:lightwood-2822:Loss @ epoch 1049: 0.12780311703681946
INFO:lightwood-2822:Loss @ epoch 1050: 0.12772810459136963
INFO:lightwood-2822:Loss @ epoch 1051: 0.1274404674768448
INFO:lightwood-2822:Loss @ epoch 1052: 0.12726184725761414
INFO:lightwood-2822:Loss @ epoch 1053: 0.1272229105234146
INFO:lightwood-2822:Loss @ epoch 1054: 0.1272438019514084
INFO:lightwood-2822:Loss @ epoch 1055: 0.12690304219722748
INFO:lightwood-2822:Loss @ epoch 1056: 0.12682569026947021
INFO:lightwood-2822:Loss @ epoch 1057: 0.1265346258878708
INFO:lightwood-2822:Loss @ epoch 1058: 0.12636567652225494
INFO:lightwood-2822:Loss @ epoch 1059: 0.12632691860198975
INFO:lightwood-2822:Loss @ epoch 1060: 0.12633532285690308
INFO:lightwood-2822:Loss @ epoch 1061: 0.1259787231683731
INFO:lightwood-2822:Loss @ epoch 1062: 0.1258990466594696
INFO:lightwood-2822:Loss @ epoch 1063: 0.12561126053333282
INFO:lightwood-2822:Loss @ epoch 1064: 0.12545344233512878
INFO:lightwood-2822:Loss @ epoch 1065: 0.12541605532169342
INFO:lightwood-2822:Loss @ epoch 1066: 0.1254201978445053
INFO:lightwood-2822:Loss @ epoch 1067: 0.1250782012939453
INFO:lightwood-2822:Loss @ epoch 1068: 0.1250016987323761
INFO:lightwood-2822:Loss @ epoch 1069: 0.1247251108288765
INFO:lightwood-2822:Loss @ epoch 1070: 0.1245650053024292
INFO:lightwood-2822:Loss @ epoch 1071: 0.12453807145357132
INFO:lightwood-2822:Loss @ epoch 1072: 0.12455521523952484
INFO:lightwood-2822:Loss @ epoch 1073: 0.12420036643743515
INFO:lightwood-2822:Loss @ epoch 1074: 0.1241225153207779
INFO:lightwood-2822:Loss @ epoch 1075: 0.12384940683841705
INFO:lightwood-2822:Loss @ epoch 1076: 0.12371952831745148
INFO:lightwood-2822:Loss @ epoch 1077: 0.12368782609701157
INFO:lightwood-2822:Loss @ epoch 1078: 0.12369099259376526
INFO:lightwood-2822:Loss @ epoch 1079: 0.12333318591117859
INFO:lightwood-2822:Loss @ epoch 1080: 0.12326589971780777
INFO:lightwood-2822:Loss @ epoch 1081: 0.12302933633327484
INFO:lightwood-2822:Loss @ epoch 1082: 0.12286393344402313
INFO:lightwood-2822:Loss @ epoch 1083: 0.12283530086278915
INFO:lightwood-2822:Loss @ epoch 1084: 0.12283457070589066
INFO:lightwood-2822:Loss @ epoch 1085: 0.12248685210943222
INFO:lightwood-2822:Loss @ epoch 1086: 0.12241316586732864
INFO:lightwood-2822:Loss @ epoch 1087: 0.12216758728027344
INFO:lightwood-2822:Loss @ epoch 1088: 0.12201714515686035
INFO:lightwood-2822:Loss @ epoch 1089: 0.12196315079927444
INFO:lightwood-2822:Loss @ epoch 1090: 0.12197311222553253
INFO:lightwood-2822:Loss @ epoch 1091: 0.12163711339235306
INFO:lightwood-2822:Loss @ epoch 1092: 0.12156134098768234
INFO:lightwood-2822:Loss @ epoch 1093: 0.12133288383483887
INFO:lightwood-2822:Loss @ epoch 1094: 0.12120620161294937
INFO:lightwood-2822:Loss @ epoch 1095: 0.12118931114673615
INFO:lightwood-2822:Loss @ epoch 1096: 0.12120679020881653
INFO:lightwood-2822:Loss @ epoch 1097: 0.12085940688848495
INFO:lightwood-2822:Loss @ epoch 1098: 0.1207910105586052
INFO:lightwood-2822:Loss @ epoch 1099: 0.12056872248649597
INFO:lightwood-2822:Loss @ epoch 1100: 0.120425745844841
INFO:lightwood-2822:Loss @ epoch 1101: 0.12040755152702332
INFO:lightwood-2822:Loss @ epoch 1102: 0.1204281747341156
INFO:lightwood-2822:Loss @ epoch 1103: 0.12007573992013931
INFO:lightwood-2822:Loss @ epoch 1104: 0.12001505494117737
INFO:lightwood-2822:Loss @ epoch 1105: 0.11979246139526367
INFO:lightwood-2822:Loss @ epoch 1106: 0.11963880062103271
INFO:lightwood-2822:Loss @ epoch 1107: 0.11962927877902985
INFO:lightwood-2822:Loss @ epoch 1108: 0.11964999884366989
INFO:lightwood-2822:Loss @ epoch 1109: 0.11930649727582932
INFO:lightwood-2822:Loss @ epoch 1110: 0.1192251443862915
INFO:lightwood-2822:Loss @ epoch 1111: 0.11898237466812134
INFO:lightwood-2822:Loss @ epoch 1112: 0.11887958645820618
INFO:lightwood-2822:Loss @ epoch 1113: 0.11886607855558395
INFO:lightwood-2822:Loss @ epoch 1114: 0.11885704100131989
INFO:lightwood-2822:Loss @ epoch 1115: 0.11851585656404495
INFO:lightwood-2822:Loss @ epoch 1116: 0.11844008415937424
INFO:lightwood-2822:Loss @ epoch 1117: 0.11821064352989197
INFO:lightwood-2822:Loss @ epoch 1118: 0.11806373298168182
INFO:lightwood-2822:Loss @ epoch 1119: 0.11804379522800446
INFO:lightwood-2822:Loss @ epoch 1120: 0.11807935684919357
INFO:lightwood-2822:Loss @ epoch 1121: 0.11774251610040665
INFO:lightwood-2822:Loss @ epoch 1122: 0.11766526848077774
INFO:lightwood-2822:Loss @ epoch 1123: 0.11743215471506119
INFO:lightwood-2822:Loss @ epoch 1124: 0.11727788299322128
INFO:lightwood-2822:Loss @ epoch 1125: 0.11728505790233612
INFO:lightwood-2822:Loss @ epoch 1126: 0.11731799691915512
INFO:lightwood-2822:Loss @ epoch 1127: 0.1169673427939415
INFO:lightwood-2822:Loss @ epoch 1128: 0.11686811596155167
INFO:lightwood-2822:Loss @ epoch 1129: 0.11662962287664413
INFO:lightwood-2822:Loss @ epoch 1130: 0.11653207242488861
INFO:lightwood-2822:Loss @ epoch 1131: 0.11653810739517212
INFO:lightwood-2822:Loss @ epoch 1132: 0.11655937135219574
INFO:lightwood-2822:Loss @ epoch 1133: 0.11621766537427902
INFO:lightwood-2822:Loss @ epoch 1134: 0.11612796783447266
INFO:lightwood-2822:Loss @ epoch 1135: 0.11588136851787567
INFO:lightwood-2822:Loss @ epoch 1136: 0.11575522273778915
INFO:lightwood-2822:Loss @ epoch 1137: 0.11578705906867981
INFO:lightwood-2822:Loss @ epoch 1138: 0.11583595722913742
INFO:lightwood-2822:Loss @ epoch 1: 0.0756365105509758
INFO:lightwood-2822:Loss @ epoch 2: 0.0723805844783783
INFO:lightwood-2822:Loss @ epoch 3: 0.06836001351475715
INFO:lightwood-2822:Loss @ epoch 4: 0.07548346221446992
INFO:lightwood-2822:Loss @ epoch 5: 0.0704605296254158
INFO:lightwood-2822:Loss @ epoch 6: 0.06740856543183327
INFO:lightwood-2822:Loss @ epoch 7: 0.07064123526215553
DEBUG:lightwood-2822: `fit_mixer` runtime: 13.21 seconds
INFO:lightwood-2822:Started fitting LGBM model
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:177: UserWarning: Found `num_iterations` in params. Will use it instead of argument
  _log_warning(f"Found `{alias}` in params. Will use it instead of argument")
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:240: UserWarning: 'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. Pass 'log_evaluation()' callback via 'callbacks' argument instead.
  _log_warning("'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. "
INFO:lightwood-2822:A single GBM iteration takes 0.1 seconds
INFO:lightwood-2822:Training GBM (<module 'lightgbm' from '/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/__init__.py'>) with 175 iterations given 21.973300342559813 seconds constraint
INFO:lightwood-2822:Lightgbm model contains 600 weak estimators
INFO:lightwood-2822:Updating lightgbm model with 7.0 iterations
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:177: UserWarning: Found `num_iterations` in params. Will use it instead of argument
  _log_warning(f"Found `{alias}` in params. Will use it instead of argument")
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:240: UserWarning: 'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. Pass 'log_evaluation()' callback via 'callbacks' argument instead.
  _log_warning("'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. "
INFO:lightwood-2822:Model now has a total of 600 weak estimators
DEBUG:lightwood-2822: `fit_mixer` runtime: 0.11 seconds
WARNING:lightwood-2822:Exception: Unspported categorical type for regression when training mixer: <lightwood.mixer.regression.Regression object at 0x7efc27ea7ee0>
INFO:lightwood-2822:Ensembling the mixer
INFO:lightwood-2822:Mixer: Neural got accuracy: 0.9222222222222223
INFO:lightwood-2822:Mixer: LightGBM got accuracy: 1.0
INFO:lightwood-2822:Picked best mixer: LightGBM
DEBUG:lightwood-2822: `fit` runtime: 13.5 seconds
INFO:lightwood-2822:[Learn phase 7/8] - Ensemble analysis
INFO:lightwood-2822:Analyzing the ensemble of mixers
INFO:lightwood-2822:The block ModelCorrelationHeatmap is now running its analyze() method
INFO:lightwood-2822:The block ICP is now running its analyze() method
INFO:lightwood-2822:The block AccStats is now running its analyze() method
INFO:lightwood-2822:The block ConfStats is now running its analyze() method
DEBUG:lightwood-2822: `analyze_ensemble` runtime: 0.06 seconds
INFO:lightwood-2822:[Learn phase 8/8] - Adjustment on validation requested
INFO:lightwood-2822:Updating the mixers
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:115: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
  warnings.warn("torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.")
INFO:lightwood-2822:Loss @ epoch 1: 0.07660853862762451
INFO:lightwood-2822:Loss @ epoch 2: 0.07766769081354141
INFO:lightwood-2822:Loss @ epoch 3: 0.07538455948233605
INFO:lightwood-2822:Loss @ epoch 4: 0.07901818603277207
INFO:lightwood-2822:Loss @ epoch 5: 0.07835438176989555
INFO:lightwood-2822:Updating lightgbm model with 7.0 iterations
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:177: UserWarning: Found `num_iterations` in params. Will use it instead of argument
  _log_warning(f"Found `{alias}` in params. Will use it instead of argument")
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/lightgbm/engine.py:240: UserWarning: 'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. Pass 'log_evaluation()' callback via 'callbacks' argument instead.
  _log_warning("'verbose_eval' argument is deprecated and will be removed in a future release of LightGBM. "
INFO:lightwood-2822:Model now has a total of 600 weak estimators
DEBUG:lightwood-2822: `adjust` runtime: 0.07 seconds
DEBUG:lightwood-2822: `learn` runtime: 13.9 seconds

Finally, we can visualize the mixer correlation matrix:

[10]:
import matplotlib.pyplot as plt
import numpy as np

mc = predictor.runtime_analyzer['mixer_correlation']  # newly produced insight

mixer_names = [c.__class__.__name__ for c in predictor.ensemble.mixers]

# plotting code
fig, ax = plt.subplots()
im = ax.imshow(mc, cmap='seismic')

# set ticks
ax.set_xticks(np.arange(mc.shape[0]))
ax.set_yticks(np.arange(mc.shape[1]))

# set tick labels
ax.set_xticklabels(mixer_names)
ax.set_yticklabels(mixer_names)

# show cell values
for i in range(len(mixer_names)):
    for j in range(len(mixer_names)):
        text = ax.text(j, i, round(mc[i, j], 3), ha="center", va="center", color="w")

../../_images/tutorials_custom_explainer_custom_explainer_20_0.png

Nice! We’ve just added an additional piece of insight regarding the predictor that Lightwood came up with for the task of predicting the Human Development Index of any given country.

What this matrix is telling us is whether the predictions of both mixers stored in the ensemble – Neural and LightGBM – have a high correlation or not.

This is, of course, a very simple example, but it shows the convenience of such an abstraction within the broader pipeline that Lightwood automates.

For more complex examples, you can check out any of the three core analysis blocks that we use:

  • lightwood.analysis.nc.calibrate.ICP

  • lightwood.analysis.helpers.acc_stats.AccStats

  • lightwood.analysis.helpers.feature_importance.GlobalFeatureImportance

[ ]: