Introduction

In this tutorial, we will go through an example to update a preexisting model. This might be useful when you come across additional data that you would want to consider, without having to train a model from scratch.

The main abstraction that Lightwood offers for this is the BaseMixer.partial_fit() method. To call it, you need to pass new training data and a held-out dev subset for internal mixer usage (e.g. early stopping). If you are using an aggregate ensemble, it’s likely you will want to do this for every single mixer. The convienient PredictorInterface.adjust() does this automatically for you.

Initial model training

First, let’s train a Lightwood predictor for the concrete strength dataset:

[1]:
from lightwood.api.high_level import ProblemDefinition, json_ai_from_problem, predictor_from_json_ai
import pandas as pd
INFO:lightwood-2548:No torchvision detected, image helpers not supported.
INFO:lightwood-2548:No torchvision/pillow detected, image encoder not supported
[2]:
# Load data
df = pd.read_csv('https://raw.githubusercontent.com/mindsdb/lightwood/staging/tests/data/concrete_strength.csv')

df = df.sample(frac=1, random_state=1)
train_df = df[:int(0.1*len(df))]
update_df = df[int(0.1*len(df)):int(0.8*len(df))]
test_df = df[int(0.8*len(df)):]

print(f'Train dataframe shape: {train_df.shape}')
print(f'Update dataframe shape: {update_df.shape}')
print(f'Test dataframe shape: {test_df.shape}')
Train dataframe shape: (103, 10)
Update dataframe shape: (721, 10)
Test dataframe shape: (206, 10)

Note that we have three different data splits.

We will use the training split for the initial model training. As you can see, it’s only a 20% of the total data we have. The update split will be used as training data to adjust/update our model. Finally, the held out test set will give us a rough idea of the impact our updating procedure has on the model’s predictive capabilities.

[3]:
# Define predictive task and predictor
target = 'concrete_strength'
pdef = ProblemDefinition.from_dict({'target': target, 'time_aim': 200})
jai = json_ai_from_problem(df, pdef)

# We will keep the architecture simple: a single neural mixer, and a `BestOf` ensemble:
jai.model = {
    "module": "BestOf",
    "args": {
        "args": "$pred_args",
        "accuracy_functions": "$accuracy_functions",
        "submodels": [{
            "module": "Neural",
            "args": {
                "fit_on_dev": False,
                "stop_after": "$problem_definition.seconds_per_mixer",
                "search_hyperparameters": False,
            }
        }]
    }
}

# Build and train the predictor
predictor = predictor_from_json_ai(jai)
predictor.learn(train_df)
INFO:lightwood-2548:Analyzing a sample of 979
INFO:lightwood-2548:from a total population of 1030, this is equivalent to 95.0% of your data.
INFO:lightwood-2548:Infering type for: id
INFO:lightwood-2548:Column id has data type integer
INFO:lightwood-2548:Infering type for: cement
INFO:lightwood-2548:Column cement has data type float
INFO:lightwood-2548:Infering type for: slag
INFO:lightwood-2548:Column slag has data type float
INFO:lightwood-2548:Infering type for: flyAsh
INFO:lightwood-2548:Column flyAsh has data type float
INFO:lightwood-2548:Infering type for: water
INFO:lightwood-2548:Column water has data type float
INFO:lightwood-2548:Infering type for: superPlasticizer
INFO:lightwood-2548:Column superPlasticizer has data type float
INFO:lightwood-2548:Infering type for: coarseAggregate
INFO:lightwood-2548:Column coarseAggregate has data type float
INFO:lightwood-2548:Infering type for: fineAggregate
INFO:lightwood-2548:Column fineAggregate has data type float
INFO:lightwood-2548:Infering type for: age
INFO:lightwood-2548:Column age has data type integer
INFO:lightwood-2548:Infering type for: concrete_strength
INFO:lightwood-2548:Column concrete_strength has data type float
INFO:lightwood-2548:Starting statistical analysis
INFO:lightwood-2548:Finished statistical analysis
INFO:lightwood-2548:Unable to import black formatter, predictor code might be a bit ugly.
INFO:lightwood-2548:[Learn phase 1/8] - Statistical analysis
INFO:lightwood-2548:Starting statistical analysis
INFO:lightwood-2548:Finished statistical analysis
DEBUG:lightwood-2548: `analyze_data` runtime: 0.02 seconds
INFO:lightwood-2548:[Learn phase 2/8] - Data preprocessing
INFO:lightwood-2548:Cleaning the data
DEBUG:lightwood-2548: `preprocess` runtime: 0.01 seconds
INFO:lightwood-2548:[Learn phase 3/8] - Data splitting
INFO:lightwood-2548:Splitting the data into train/test
DEBUG:lightwood-2548: `split` runtime: 0.0 seconds
INFO:lightwood-2548:[Learn phase 4/8] - Preparing encoders
INFO:lightwood-2548:Done running for: id
INFO:lightwood-2548:Done running for: cement
INFO:lightwood-2548:Done running for: slag
INFO:lightwood-2548:Done running for: flyAsh
INFO:lightwood-2548:Done running for: water
INFO:lightwood-2548:Done running for: superPlasticizer
INFO:lightwood-2548:Done running for: coarseAggregate
INFO:lightwood-2548:Done running for: fineAggregate
INFO:lightwood-2548:Done running for: age
DEBUG:lightwood-2548: `prepare` runtime: 0.1 seconds
INFO:lightwood-2548:[Learn phase 5/8] - Feature generation
INFO:lightwood-2548:Featurizing the data
DEBUG:lightwood-2548: `featurize` runtime: 0.0 seconds
INFO:lightwood-2548:[Learn phase 6/8] - Mixer training
INFO:lightwood-2548:Training the mixers
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:115: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
  warnings.warn("torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.")
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/pytorch_ranger/ranger.py:172: UserWarning: This overload of addcmul_ is deprecated:
        addcmul_(Number value, Tensor tensor1, Tensor tensor2)
Consider using one of the following signatures instead:
        addcmul_(Tensor tensor1, Tensor tensor2, *, Number value) (Triggered internally at  ../torch/csrc/utils/python_arg_parser.cpp:1025.)
  exp_avg_sq.mul_(beta2).addcmul_(1 - beta2, grad, grad)
INFO:lightwood-2548:Loss of 7.153866291046143 with learning rate 0.0001
INFO:lightwood-2548:Loss of 6.8837056159973145 with learning rate 0.00014
INFO:lightwood-2548:Loss of 7.064479351043701 with learning rate 0.00019599999999999997
INFO:lightwood-2548:Found learning rate of: 0.00014
INFO:lightwood-2548:Loss @ epoch 1: 3.162095546722412
INFO:lightwood-2548:Loss @ epoch 2: 3.4808480739593506
INFO:lightwood-2548:Loss @ epoch 3: 3.4468672275543213
INFO:lightwood-2548:Loss @ epoch 4: 3.4402384757995605
INFO:lightwood-2548:Loss @ epoch 5: 3.3827364444732666
INFO:lightwood-2548:Loss @ epoch 6: 3.3782267570495605
INFO:lightwood-2548:Loss @ epoch 7: 3.304530143737793
INFO:lightwood-2548:Loss @ epoch 8: 3.300995111465454
INFO:lightwood-2548:Loss @ epoch 9: 3.214940071105957
INFO:lightwood-2548:Loss @ epoch 10: 3.2120182514190674
INFO:lightwood-2548:Loss @ epoch 11: 3.1162424087524414
INFO:lightwood-2548:Loss @ epoch 12: 3.1137688159942627
INFO:lightwood-2548:Loss @ epoch 13: 3.01035737991333
INFO:lightwood-2548:Loss @ epoch 14: 3.0082414150238037
INFO:lightwood-2548:Loss @ epoch 15: 2.8988308906555176
INFO:lightwood-2548:Loss @ epoch 16: 2.897016763687134
INFO:lightwood-2548:Loss @ epoch 17: 2.783118724822998
INFO:lightwood-2548:Loss @ epoch 18: 2.7815678119659424
INFO:lightwood-2548:Loss @ epoch 19: 2.6644930839538574
INFO:lightwood-2548:Loss @ epoch 20: 2.6631782054901123
INFO:lightwood-2548:Loss @ epoch 21: 2.5441019535064697
INFO:lightwood-2548:Loss @ epoch 22: 2.5429975986480713
INFO:lightwood-2548:Loss @ epoch 23: 2.422982692718506
INFO:lightwood-2548:Loss @ epoch 24: 2.4220690727233887
INFO:lightwood-2548:Loss @ epoch 25: 2.302001476287842
INFO:lightwood-2548:Loss @ epoch 26: 2.301260471343994
INFO:lightwood-2548:Loss @ epoch 27: 2.1820054054260254
INFO:lightwood-2548:Loss @ epoch 28: 2.1814208030700684
INFO:lightwood-2548:Loss @ epoch 29: 2.063715696334839
INFO:lightwood-2548:Loss @ epoch 30: 2.0632710456848145
INFO:lightwood-2548:Loss @ epoch 31: 1.9477651119232178
INFO:lightwood-2548:Loss @ epoch 32: 1.9474451541900635
INFO:lightwood-2548:Loss @ epoch 33: 1.8346989154815674
INFO:lightwood-2548:Loss @ epoch 34: 1.8344911336898804
INFO:lightwood-2548:Loss @ epoch 35: 1.724977970123291
INFO:lightwood-2548:Loss @ epoch 36: 1.7248685359954834
INFO:lightwood-2548:Loss @ epoch 37: 1.618973970413208
INFO:lightwood-2548:Loss @ epoch 38: 1.6189501285552979
INFO:lightwood-2548:Loss @ epoch 39: 1.5169769525527954
INFO:lightwood-2548:Loss @ epoch 40: 1.5170263051986694
INFO:lightwood-2548:Loss @ epoch 41: 1.4192277193069458
INFO:lightwood-2548:Loss @ epoch 42: 1.419340968132019
INFO:lightwood-2548:Loss @ epoch 43: 1.325916051864624
INFO:lightwood-2548:Loss @ epoch 44: 1.3260842561721802
INFO:lightwood-2548:Loss @ epoch 45: 1.2371675968170166
INFO:lightwood-2548:Loss @ epoch 46: 1.2373806238174438
INFO:lightwood-2548:Loss @ epoch 47: 1.1530762910842896
INFO:lightwood-2548:Loss @ epoch 48: 1.1533257961273193
INFO:lightwood-2548:Loss @ epoch 49: 1.073638677597046
INFO:lightwood-2548:Loss @ epoch 50: 1.0739187002182007
INFO:lightwood-2548:Loss @ epoch 51: 0.9987974166870117
INFO:lightwood-2548:Loss @ epoch 52: 0.999101459980011
INFO:lightwood-2548:Loss @ epoch 53: 0.928507924079895
INFO:lightwood-2548:Loss @ epoch 54: 0.928829550743103
INFO:lightwood-2548:Loss @ epoch 55: 0.8626793622970581
INFO:lightwood-2548:Loss @ epoch 56: 0.8630120754241943
INFO:lightwood-2548:Loss @ epoch 57: 0.8011684417724609
INFO:lightwood-2548:Loss @ epoch 58: 0.801507294178009
INFO:lightwood-2548:Loss @ epoch 59: 0.7438275218009949
INFO:lightwood-2548:Loss @ epoch 60: 0.7441690564155579
INFO:lightwood-2548:Loss @ epoch 61: 0.6904723644256592
INFO:lightwood-2548:Loss @ epoch 62: 0.6908120512962341
INFO:lightwood-2548:Loss @ epoch 63: 0.6409392952919006
INFO:lightwood-2548:Loss @ epoch 64: 0.6412740349769592
INFO:lightwood-2548:Loss @ epoch 65: 0.5950359106063843
INFO:lightwood-2548:Loss @ epoch 66: 0.5953627228736877
INFO:lightwood-2548:Loss @ epoch 67: 0.5525646209716797
INFO:lightwood-2548:Loss @ epoch 68: 0.5528810024261475
INFO:lightwood-2548:Loss @ epoch 69: 0.513323187828064
INFO:lightwood-2548:Loss @ epoch 70: 0.5136263966560364
INFO:lightwood-2548:Loss @ epoch 71: 0.4771132171154022
INFO:lightwood-2548:Loss @ epoch 72: 0.4774025082588196
INFO:lightwood-2548:Loss @ epoch 73: 0.44372856616973877
INFO:lightwood-2548:Loss @ epoch 74: 0.44400283694267273
INFO:lightwood-2548:Loss @ epoch 75: 0.41296884417533875
INFO:lightwood-2548:Loss @ epoch 76: 0.4132263958454132
INFO:lightwood-2548:Loss @ epoch 77: 0.3846345543861389
INFO:lightwood-2548:Loss @ epoch 78: 0.38487595319747925
INFO:lightwood-2548:Loss @ epoch 79: 0.358543336391449
INFO:lightwood-2548:Loss @ epoch 80: 0.3587690591812134
INFO:lightwood-2548:Loss @ epoch 81: 0.33454322814941406
INFO:lightwood-2548:Loss @ epoch 82: 0.3347531259059906
INFO:lightwood-2548:Loss @ epoch 83: 0.31246522068977356
INFO:lightwood-2548:Loss @ epoch 84: 0.3126596212387085
INFO:lightwood-2548:Loss @ epoch 85: 0.2921428084373474
INFO:lightwood-2548:Loss @ epoch 86: 0.29232269525527954
INFO:lightwood-2548:Loss @ epoch 87: 0.27343931794166565
INFO:lightwood-2548:Loss @ epoch 88: 0.27360522747039795
INFO:lightwood-2548:Loss @ epoch 89: 0.2562256157398224
INFO:lightwood-2548:Loss @ epoch 90: 0.2563779950141907
INFO:lightwood-2548:Loss @ epoch 91: 0.24037514626979828
INFO:lightwood-2548:Loss @ epoch 92: 0.24051493406295776
INFO:lightwood-2548:Loss @ epoch 93: 0.22578652203083038
INFO:lightwood-2548:Loss @ epoch 94: 0.22591444849967957
INFO:lightwood-2548:Loss @ epoch 95: 0.21234583854675293
INFO:lightwood-2548:Loss @ epoch 96: 0.2124629020690918
INFO:lightwood-2548:Loss @ epoch 97: 0.19996081292629242
INFO:lightwood-2548:Loss @ epoch 98: 0.2000676691532135
INFO:lightwood-2548:Loss @ epoch 99: 0.18854695558547974
INFO:lightwood-2548:Loss @ epoch 100: 0.18864423036575317
INFO:lightwood-2548:Loss @ epoch 101: 0.17803452908992767
INFO:lightwood-2548:Loss @ epoch 102: 0.17812293767929077
INFO:lightwood-2548:Loss @ epoch 103: 0.16833838820457458
INFO:lightwood-2548:Loss @ epoch 104: 0.16841872036457062
INFO:lightwood-2548:Loss @ epoch 105: 0.15939119458198547
INFO:lightwood-2548:Loss @ epoch 106: 0.15946407616138458
INFO:lightwood-2548:Loss @ epoch 107: 0.1511337161064148
INFO:lightwood-2548:Loss @ epoch 108: 0.151199609041214
INFO:lightwood-2548:Loss @ epoch 109: 0.1435091346502304
INFO:lightwood-2548:Loss @ epoch 110: 0.14356893301010132
INFO:lightwood-2548:Loss @ epoch 111: 0.13646595180034637
INFO:lightwood-2548:Loss @ epoch 112: 0.1365201324224472
INFO:lightwood-2548:Loss @ epoch 113: 0.12995712459087372
INFO:lightwood-2548:Loss @ epoch 114: 0.13000629842281342
INFO:lightwood-2548:Loss @ epoch 115: 0.12393999099731445
INFO:lightwood-2548:Loss @ epoch 116: 0.12398432940244675
INFO:lightwood-2548:Loss @ epoch 117: 0.11837441474199295
INFO:lightwood-2548:Loss @ epoch 118: 0.11841466277837753
INFO:lightwood-2548:Loss @ epoch 119: 0.11322391778230667
INFO:lightwood-2548:Loss @ epoch 120: 0.11326029896736145
INFO:lightwood-2548:Loss @ epoch 121: 0.10845529288053513
INFO:lightwood-2548:Loss @ epoch 122: 0.10848837345838547
INFO:lightwood-2548:Loss @ epoch 123: 0.10403768718242645
INFO:lightwood-2548:Loss @ epoch 124: 0.10406771302223206
INFO:lightwood-2548:Loss @ epoch 125: 0.0999443382024765
INFO:lightwood-2548:Loss @ epoch 126: 0.09997159987688065
INFO:lightwood-2548:Loss @ epoch 127: 0.0961480438709259
INFO:lightwood-2548:Loss @ epoch 128: 0.096172995865345
INFO:lightwood-2548:Loss @ epoch 129: 0.0926254466176033
INFO:lightwood-2548:Loss @ epoch 130: 0.09264825284481049
INFO:lightwood-2548:Loss @ epoch 131: 0.08935500681400299
INFO:lightwood-2548:Loss @ epoch 132: 0.08937594294548035
INFO:lightwood-2548:Loss @ epoch 133: 0.08631822466850281
INFO:lightwood-2548:Loss @ epoch 134: 0.08633731305599213
INFO:lightwood-2548:Loss @ epoch 135: 0.08349747955799103
INFO:lightwood-2548:Loss @ epoch 136: 0.08351494371891022
INFO:lightwood-2548:Loss @ epoch 137: 0.08087345957756042
INFO:lightwood-2548:Loss @ epoch 138: 0.0808895006775856
INFO:lightwood-2548:Loss @ epoch 139: 0.07843017578125
INFO:lightwood-2548:Loss @ epoch 140: 0.0784449502825737
INFO:lightwood-2548:Loss @ epoch 141: 0.07615318894386292
INFO:lightwood-2548:Loss @ epoch 142: 0.0761668011546135
INFO:lightwood-2548:Loss @ epoch 143: 0.0740291029214859
INFO:lightwood-2548:Loss @ epoch 144: 0.07404157519340515
INFO:lightwood-2548:Loss @ epoch 145: 0.07204649597406387
INFO:lightwood-2548:Loss @ epoch 146: 0.07205799967050552
INFO:lightwood-2548:Loss @ epoch 147: 0.07019531726837158
INFO:lightwood-2548:Loss @ epoch 148: 0.07020591199398041
INFO:lightwood-2548:Loss @ epoch 149: 0.06846503913402557
INFO:lightwood-2548:Loss @ epoch 150: 0.06847492605447769
INFO:lightwood-2548:Loss @ epoch 151: 0.06684551388025284
INFO:lightwood-2548:Loss @ epoch 152: 0.0668545812368393
INFO:lightwood-2548:Loss @ epoch 153: 0.06532759964466095
INFO:lightwood-2548:Loss @ epoch 154: 0.06533605605363846
INFO:lightwood-2548:Loss @ epoch 155: 0.06390313804149628
INFO:lightwood-2548:Loss @ epoch 156: 0.06391094624996185
INFO:lightwood-2548:Loss @ epoch 157: 0.0625658631324768
INFO:lightwood-2548:Loss @ epoch 158: 0.06257309764623642
INFO:lightwood-2548:Loss @ epoch 159: 0.061308957636356354
INFO:lightwood-2548:Loss @ epoch 160: 0.0613156333565712
INFO:lightwood-2548:Loss @ epoch 161: 0.06012523174285889
INFO:lightwood-2548:Loss @ epoch 162: 0.06013139709830284
INFO:lightwood-2548:Loss @ epoch 163: 0.05900917947292328
INFO:lightwood-2548:Loss @ epoch 164: 0.05901492387056351
INFO:lightwood-2548:Loss @ epoch 165: 0.05795574560761452
INFO:lightwood-2548:Loss @ epoch 166: 0.05796102061867714
INFO:lightwood-2548:Loss @ epoch 167: 0.056960176676511765
INFO:lightwood-2548:Loss @ epoch 168: 0.056965067982673645
INFO:lightwood-2548:Loss @ epoch 169: 0.056018222123384476
INFO:lightwood-2548:Loss @ epoch 170: 0.05602269247174263
INFO:lightwood-2548:Loss @ epoch 171: 0.05512585490942001
INFO:lightwood-2548:Loss @ epoch 172: 0.055129941552877426
INFO:lightwood-2548:Loss @ epoch 173: 0.054279521107673645
INFO:lightwood-2548:Loss @ epoch 174: 0.054283302277326584
INFO:lightwood-2548:Loss @ epoch 175: 0.053475916385650635
INFO:lightwood-2548:Loss @ epoch 176: 0.05347947031259537
INFO:lightwood-2548:Loss @ epoch 177: 0.05271228402853012
INFO:lightwood-2548:Loss @ epoch 178: 0.05271557718515396
INFO:lightwood-2548:Loss @ epoch 179: 0.05198604613542557
INFO:lightwood-2548:Loss @ epoch 180: 0.05198908597230911
INFO:lightwood-2548:Loss @ epoch 181: 0.05129483714699745
INFO:lightwood-2548:Loss @ epoch 182: 0.05129767954349518
INFO:lightwood-2548:Loss @ epoch 183: 0.05063660815358162
INFO:lightwood-2548:Loss @ epoch 184: 0.05063919350504875
INFO:lightwood-2548:Loss @ epoch 185: 0.05000946670770645
INFO:lightwood-2548:Loss @ epoch 186: 0.050011951476335526
INFO:lightwood-2548:Loss @ epoch 187: 0.049411747604608536
INFO:lightwood-2548:Loss @ epoch 188: 0.04941414296627045
INFO:lightwood-2548:Loss @ epoch 189: 0.0488419346511364
INFO:lightwood-2548:Loss @ epoch 190: 0.04884418100118637
INFO:lightwood-2548:Loss @ epoch 191: 0.04829869791865349
INFO:lightwood-2548:Loss @ epoch 192: 0.04830075800418854
INFO:lightwood-2548:Loss @ epoch 193: 0.04778038710355759
INFO:lightwood-2548:Loss @ epoch 194: 0.04778226837515831
INFO:lightwood-2548:Loss @ epoch 195: 0.04728569835424423
INFO:lightwood-2548:Loss @ epoch 196: 0.0472874790430069
INFO:lightwood-2548:Loss @ epoch 197: 0.046813998371362686
INFO:lightwood-2548:Loss @ epoch 198: 0.046815671026706696
INFO:lightwood-2548:Loss @ epoch 199: 0.046364251524209976
INFO:lightwood-2548:Loss @ epoch 200: 0.04636577144265175
INFO:lightwood-2548:Loss @ epoch 201: 0.04593560844659805
INFO:lightwood-2548:Loss @ epoch 202: 0.04593712463974953
INFO:lightwood-2548:Loss @ epoch 203: 0.04552730172872543
INFO:lightwood-2548:Loss @ epoch 204: 0.04552868381142616
INFO:lightwood-2548:Loss @ epoch 205: 0.0451384000480175
INFO:lightwood-2548:Loss @ epoch 206: 0.04513971507549286
INFO:lightwood-2548:Loss @ epoch 207: 0.04476826637983322
INFO:lightwood-2548:Loss @ epoch 208: 0.044769540429115295
INFO:lightwood-2548:Loss @ epoch 209: 0.04441629722714424
INFO:lightwood-2548:Loss @ epoch 210: 0.04441754147410393
INFO:lightwood-2548:Loss @ epoch 211: 0.04408185929059982
INFO:lightwood-2548:Loss @ epoch 212: 0.04408304765820503
INFO:lightwood-2548:Loss @ epoch 213: 0.043764468282461166
INFO:lightwood-2548:Loss @ epoch 214: 0.04376551881432533
INFO:lightwood-2548:Loss @ epoch 215: 0.04346352443099022
INFO:lightwood-2548:Loss @ epoch 216: 0.04346450790762901
INFO:lightwood-2548:Loss @ epoch 217: 0.04317828640341759
INFO:lightwood-2548:Loss @ epoch 218: 0.04317918047308922
INFO:lightwood-2548:Loss @ epoch 219: 0.04290812835097313
INFO:lightwood-2548:Loss @ epoch 220: 0.04290897026658058
INFO:lightwood-2548:Loss @ epoch 221: 0.042652640491724014
INFO:lightwood-2548:Loss @ epoch 222: 0.042653489857912064
INFO:lightwood-2548:Loss @ epoch 223: 0.042411498725414276
INFO:lightwood-2548:Loss @ epoch 224: 0.04241231828927994
INFO:lightwood-2548:Loss @ epoch 225: 0.04218415915966034
INFO:lightwood-2548:Loss @ epoch 226: 0.04218496009707451
INFO:lightwood-2548:Loss @ epoch 227: 0.041970327496528625
INFO:lightwood-2548:Loss @ epoch 228: 0.04197108745574951
INFO:lightwood-2548:Loss @ epoch 229: 0.04176963493227959
INFO:lightwood-2548:Loss @ epoch 230: 0.0417703315615654
INFO:lightwood-2548:Loss @ epoch 231: 0.04158157855272293
INFO:lightwood-2548:Loss @ epoch 232: 0.04158216714859009
INFO:lightwood-2548:Loss @ epoch 233: 0.04140692576766014
INFO:lightwood-2548:Loss @ epoch 234: 0.04140745475888252
INFO:lightwood-2548:Loss @ epoch 235: 0.04124433174729347
INFO:lightwood-2548:Loss @ epoch 236: 0.0412447452545166
INFO:lightwood-2548:Loss @ epoch 237: 0.04109252244234085
INFO:lightwood-2548:Loss @ epoch 238: 0.04109293967485428
INFO:lightwood-2548:Loss @ epoch 239: 0.04095109924674034
INFO:lightwood-2548:Loss @ epoch 240: 0.04095141589641571
INFO:lightwood-2548:Loss @ epoch 241: 0.04081958159804344
INFO:lightwood-2548:Loss @ epoch 242: 0.040819842368364334
INFO:lightwood-2548:Loss @ epoch 243: 0.040697574615478516
INFO:lightwood-2548:Loss @ epoch 244: 0.040697868913412094
INFO:lightwood-2548:Loss @ epoch 245: 0.04058486595749855
INFO:lightwood-2548:Loss @ epoch 246: 0.0405849888920784
INFO:lightwood-2548:Loss @ epoch 247: 0.040480710566043854
INFO:lightwood-2548:Loss @ epoch 248: 0.040480826050043106
INFO:lightwood-2548:Loss @ epoch 249: 0.040384870022535324
INFO:lightwood-2548:Loss @ epoch 250: 0.0403849221765995
INFO:lightwood-2548:Loss @ epoch 251: 0.04029689356684685
INFO:lightwood-2548:Loss @ epoch 252: 0.04029691964387894
INFO:lightwood-2548:Loss @ epoch 253: 0.04021612927317619
INFO:lightwood-2548:Loss @ epoch 254: 0.04021613299846649
INFO:lightwood-2548:Loss @ epoch 255: 0.04014238342642784
INFO:lightwood-2548:Loss @ epoch 256: 0.04014241322875023
INFO:lightwood-2548:Loss @ epoch 257: 0.040075670927762985
INFO:lightwood-2548:Loss @ epoch 258: 0.04007556289434433
INFO:lightwood-2548:Loss @ epoch 259: 0.04001564905047417
INFO:lightwood-2548:Loss @ epoch 260: 0.040015529841184616
INFO:lightwood-2548:Loss @ epoch 261: 0.039961811155080795
INFO:lightwood-2548:Loss @ epoch 262: 0.03996160998940468
INFO:lightwood-2548:Loss @ epoch 263: 0.039913590997457504
INFO:lightwood-2548:Loss @ epoch 264: 0.03991340100765228
INFO:lightwood-2548:Loss @ epoch 265: 0.039870548993349075
INFO:lightwood-2548:Loss @ epoch 266: 0.039870209991931915
INFO:lightwood-2548:Loss @ epoch 267: 0.039832066744565964
INFO:lightwood-2548:Loss @ epoch 268: 0.03983166441321373
INFO:lightwood-2548:Loss @ epoch 269: 0.039797887206077576
INFO:lightwood-2548:Loss @ epoch 270: 0.03979744762182236
INFO:lightwood-2548:Loss @ epoch 271: 0.039768002927303314
INFO:lightwood-2548:Loss @ epoch 272: 0.03976759687066078
INFO:lightwood-2548:Loss @ epoch 273: 0.03974192589521408
INFO:lightwood-2548:Loss @ epoch 274: 0.03974149748682976
INFO:lightwood-2548:Loss @ epoch 275: 0.03971996530890465
INFO:lightwood-2548:Loss @ epoch 276: 0.03971938416361809
INFO:lightwood-2548:Loss @ epoch 277: 0.03970235586166382
INFO:lightwood-2548:Loss @ epoch 278: 0.039701830595731735
INFO:lightwood-2548:Loss @ epoch 279: 0.03968769684433937
INFO:lightwood-2548:Loss @ epoch 280: 0.03968712314963341
INFO:lightwood-2548:Loss @ epoch 281: 0.03967548534274101
INFO:lightwood-2548:Loss @ epoch 282: 0.03967494145035744
INFO:lightwood-2548:Loss @ epoch 283: 0.039665646851062775
INFO:lightwood-2548:Loss @ epoch 284: 0.03966507688164711
INFO:lightwood-2548:Loss @ epoch 285: 0.03965789079666138
INFO:lightwood-2548:Loss @ epoch 286: 0.03965733200311661
INFO:lightwood-2548:Loss @ epoch 287: 0.03965215012431145
INFO:lightwood-2548:Loss @ epoch 288: 0.03965151309967041
INFO:lightwood-2548:Loss @ epoch 289: 0.03964811936020851
INFO:lightwood-2548:Loss @ epoch 290: 0.03964753448963165
INFO:lightwood-2548:Loss @ epoch 291: 0.0396457239985466
INFO:lightwood-2548:Loss @ epoch 292: 0.039645079523324966
INFO:lightwood-2548:Loss @ epoch 293: 0.03964453563094139
INFO:lightwood-2548:Loss @ epoch 294: 0.03964388370513916
INFO:lightwood-2548:Loss @ epoch 295: 0.03964449092745781
DEBUG:lightwood-2548: `fit_mixer` runtime: 2.98 seconds
INFO:lightwood-2548:Ensembling the mixer
INFO:lightwood-2548:Mixer: Neural got accuracy: 0.2812425852809247
INFO:lightwood-2548:Picked best mixer: Neural
DEBUG:lightwood-2548: `fit` runtime: 3.08 seconds
INFO:lightwood-2548:[Learn phase 7/8] - Ensemble analysis
INFO:lightwood-2548:Analyzing the ensemble of mixers
INFO:lightwood-2548:The block ICP is now running its analyze() method
INFO:lightwood-2548:The block AccStats is now running its analyze() method
INFO:lightwood-2548:The block ConfStats is now running its analyze() method
DEBUG:lightwood-2548: `analyze_ensemble` runtime: 0.07 seconds
INFO:lightwood-2548:[Learn phase 8/8] - Adjustment on validation requested
INFO:lightwood-2548:Updating the mixers
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:115: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
  warnings.warn("torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.")
INFO:lightwood-2548:Loss @ epoch 1: 0.09710239991545677
INFO:lightwood-2548:Loss @ epoch 2: 0.06753953918814659
INFO:lightwood-2548:Loss @ epoch 3: 0.08631765718261401
INFO:lightwood-2548:Loss @ epoch 4: 0.06293793395161629
INFO:lightwood-2548:Loss @ epoch 5: 0.07065049062172572
INFO:lightwood-2548:Loss @ epoch 6: 0.053511256662507854
INFO:lightwood-2548:Loss @ epoch 7: 0.06612508992354076
DEBUG:lightwood-2548: `adjust` runtime: 0.04 seconds
DEBUG:lightwood-2548: `learn` runtime: 3.35 seconds
[4]:
# Train and get predictions for the held out test set
predictions = predictor.predict(test_df)
predictions
INFO:lightwood-2548:[Predict phase 1/4] - Data preprocessing
INFO:lightwood-2548:Cleaning the data
DEBUG:lightwood-2548: `preprocess` runtime: 0.01 seconds
INFO:lightwood-2548:[Predict phase 2/4] - Feature generation
INFO:lightwood-2548:Featurizing the data
DEBUG:lightwood-2548: `featurize` runtime: 0.0 seconds
INFO:lightwood-2548:[Predict phase 3/4] - Calling ensemble
INFO:lightwood-2548:[Predict phase 4/4] - Analyzing output
INFO:lightwood-2548:The block ICP is now running its explain() method
INFO:lightwood-2548:The block AccStats is now running its explain() method
INFO:lightwood-2548:AccStats.explain() has not been implemented, no modifications will be done to the data insights.
INFO:lightwood-2548:The block ConfStats is now running its explain() method
INFO:lightwood-2548:ConfStats.explain() has not been implemented, no modifications will be done to the data insights.
DEBUG:lightwood-2548: `predict` runtime: 1.78 seconds
[4]:
original_index prediction confidence lower upper
0 0 49.177926 0.9991 0.0 99.741836
1 1 18.238433 0.9991 0.0 68.802343
2 2 22.289931 0.9991 0.0 72.853840
3 3 20.362166 0.9991 0.0 70.926075
4 4 38.186172 0.9991 0.0 88.750082
... ... ... ... ... ...
201 201 47.799281 0.9991 0.0 98.363190
202 202 41.190282 0.9991 0.0 91.754192
203 203 37.798300 0.9991 0.0 88.362210
204 204 29.786588 0.9991 0.0 80.350498
205 205 34.855963 0.9991 0.0 85.419873

206 rows × 5 columns

Updating the predictor

For this, we have two options:

BaseMixer.partial_fit()

Updates a single mixer. You need to pass the new data wrapped in EncodedDs objects.

Arguments: * train_data: EncodedDs * dev_data: EncodedDs

If the mixer does not need a dev_data partition, pass a dummy:

dev_data = EncodedDs(predictor.encoders, pd.DataFrame(), predictor.target)

PredictorInterface.adjust()

Updates all mixers inside the predictor by calling their respective partial_fit() methods.

Arguments: * new_data: Union[EncodedDs, ConcatedEncodedDs, pd.DataFrame] * old_data: Optional[Union[EncodedDs, ConcatedEncodedDs, pd.DataFrame]]

Let’s adjust our predictor:

[5]:
from lightwood.data import EncodedDs

train_ds = EncodedDs(predictor.encoders, train_df, target)
update_ds = EncodedDs(predictor.encoders, update_df, target)

predictor.adjust(update_ds, train_ds)  # data to adjust and original data
INFO:lightwood-2548:Updating the mixers
/opt/hostedtoolcache/Python/3.9.13/x64/lib/python3.9/site-packages/torch/cuda/amp/grad_scaler.py:115: UserWarning: torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.
  warnings.warn("torch.cuda.amp.GradScaler is enabled, but CUDA is not available.  Disabling.")
INFO:lightwood-2548:Loss @ epoch 1: 0.06666224698225658
DEBUG:lightwood-2548: `adjust` runtime: 5.46 seconds
[6]:
new_predictions = predictor.predict(test_df)
new_predictions
INFO:lightwood-2548:[Predict phase 1/4] - Data preprocessing
INFO:lightwood-2548:Cleaning the data
DEBUG:lightwood-2548: `preprocess` runtime: 0.01 seconds
INFO:lightwood-2548:[Predict phase 2/4] - Feature generation
INFO:lightwood-2548:Featurizing the data
DEBUG:lightwood-2548: `featurize` runtime: 0.0 seconds
INFO:lightwood-2548:[Predict phase 3/4] - Calling ensemble
INFO:lightwood-2548:[Predict phase 4/4] - Analyzing output
INFO:lightwood-2548:The block ICP is now running its explain() method
INFO:lightwood-2548:The block AccStats is now running its explain() method
INFO:lightwood-2548:AccStats.explain() has not been implemented, no modifications will be done to the data insights.
INFO:lightwood-2548:The block ConfStats is now running its explain() method
INFO:lightwood-2548:ConfStats.explain() has not been implemented, no modifications will be done to the data insights.
DEBUG:lightwood-2548: `predict` runtime: 1.81 seconds
[6]:
original_index prediction confidence lower upper
0 0 50.047745 0.9991 0.0 100.611655
1 1 19.834124 0.9991 0.0 70.398033
2 2 23.211976 0.9991 0.0 73.775885
3 3 21.163995 0.9991 0.0 71.727905
4 4 39.810138 0.9991 0.0 90.374048
... ... ... ... ... ...
201 201 48.718162 0.9991 0.0 99.282072
202 202 41.109587 0.9991 0.0 91.673497
203 203 39.365741 0.9991 0.0 89.929650
204 204 30.945220 0.9991 0.0 81.509130
205 205 34.835177 0.9991 0.0 85.399087

206 rows × 5 columns

Nice! Our predictor was updated, and new predictions are looking good. Let’s compare the old and new accuracies:

[7]:
from sklearn.metrics import r2_score
import numpy as np

old_acc = r2_score(test_df['concrete_strength'], predictions['prediction'])
new_acc = r2_score(test_df['concrete_strength'], new_predictions['prediction'])

print(f'Old Accuracy: {round(old_acc, 3)}\nNew Accuracy: {round(new_acc, 3)}')
Old Accuracy: 0.442
New Accuracy: 0.473

After updating, we see an increase in the R2 score of predictions for the held out test set.

Conclusion

We have gone through a simple example of how Lightwood predictors can leverage newly acquired data to improve their predictions. The interface for doing so is fairly simple, requiring only some new data and a single call to update.

You can further customize the logic for updating your mixers by modifying the partial_fit() methods in them.