Porting ML Models - ML System Complexity

As I specified in the past, the Machine Learning Models trained on PC in Python for weather classification and regression, need to be ported to plain C in order to be used in the Arduino code. So the trained classifiers get ported over using MicroMLGen as part of Eloquent ML Arduino: https://github.com/eloquentarduino/micromlgen

The system can be easily ported with a few lines of code, as shown here:

from micromlgen import port
print(port(clf))

I had run into an issue in porting the XGBoost, as specified in this Github issue, with the actual solution: https://github.com/eloquentarduino/micromlgen/issues/13

The XGboost port code uses a temporary file in APPDATA/LOCAL to create a temporary json file. There is no info about this provided to the user. In fact, tested on 3 systems, the file was not generated because the Jupyter Notebook does not have access to the APPDATA/LOCAL folder, even with admin right or by trusting the notebook, it still cannot create it.

This is the type of error generated: XGBoostError: [14:36:23] C:\Users\Administrator\workspace\xgboost-win64_release_1.0.0\dmlc-core\src\io\local_filesys.cc:209: Check failed: allow_null: LocalFileSystem::Open "C:\Users\ZW\AppData\Local\Temp\tmp_mu9qwkg": Permission denied

SOLUTION: By removing the None from: def port_xgboost(clf, tmp_file=None, **kwargs):

The user can then specify the None in their python script if they would prefer (and if it works) a temp file in APPDATA/LOCAL or they can actually specify the directory with the file ending in .json: print(port(xgb, tmp_file = "C:\\Users\\*username*\\Desktop\\test.json")))

And they can use the code exemplified for the DecisionTree/RandomForest to create a .h file:

with open('XGBoostClassifier.h', 'w') as file: file.write(port(xgb, tmp_file = "C:\\Users\\*username*\\Desktop\\test.json"))

The files get ported as .h, and can be easily imported and used in the Arduino code:

#include "src/ml-models/SVMClassifier.h"
#include "src/ml-models/XGBoostClassifier.h"
#include "src/ml-models/DecisionTreeClassifier.h"
#include "src/ml-models/GaussianNaiveBayesClassifier.h"
#include "src/ml-models/DecisionTreeRegressor.h"

Eloquent::ML::Port::SVM SVM_classifier;
Eloquent::ML::Port::XGBClassifier XGBoost_classifier;
Eloquent::ML::Port::DecisionTree DecisionTree_classifier;
Eloquent::ML::Port::GaussianNB GaussianNB_classifier;
Eloquent::ML::Port::DecisionTreeRegressor decisiontree_regressor;

void loop() {
    float X[] = {temp, feels_like_temp, dew_point, relative_humidity, sea_level_pressure};
    float X_SVM[] = {temp, feels_like_temp, dew_point, relative_humidity, sea_level_pressure, uv_index};
    
    int weather_svm = SVM_classifier.predict(X);
    int weather_xgb = XGBoost_classifier.predict(X);
    int weather_dtc = DecisionTree_classifier.predict(X);
    int weather_gnb = GaussianNB_classifier.predict(X);
    
    float rainfall_regression = decisiontree_regressor.predict(X);

    Serial.println("\nThe weather has been identified by the Support Vector Machines Classifier as: " + String(weather_svm));
    Serial.println("The weather has been identified by the XGBoost Classifier as: " + String(weather_xgb));
    Serial.println("The weather has been identified by the Decision Tree Classifier as: " + String(weather_dtc));
    Serial.println("The weather has been identified by the Gaussian Naive Bayes Classifier as: " + String(weather_gnb));
    Serial.println("The predicted amount of rainfall in millimeters in: " + String(rainfall_regression) + " mm");
    delay(5000);
}

Here is the amount of memory and flash usage:

Because of the high usage of the memory (not RAM), the models cannot be loaded on the Nicla Sense ME, therefore, the models will be loaded on the Xiao Sense, which will receive the X array through I2C, perform the classification and send the labels back:

    Y[0] = SVM_classifier.predict(X_SVM);                                                                                                                                             
    Y[1] = XGBoost_classifier.predict(X);                                                                                                                                             
    Y[2] = DecisionTree_classifier.predict(X);                                                                                                                                        
    Y[3] = GaussianNB_classifier.predict(X);                                                                                                                                          
    Y[4] = decisiontree_regressor.predict(X); 

void receiveEvent(int howMany) {
  for (int i = 0; i < 6; i++) {                                                                                                                                                       
    I2C_readAnything(X_SVM[i]);                                                                                                                                                       
    if (i < 5) X[i] = X_SVM[i];                                                                                                                                                       
  }                                                                                                                                                                                   
  
  startInferencing = true;                                                                                                                                                            
}

void requestEvent(void) {
  for (int j = 0; j < sizeof(Y); j++) I2C_writeAnything(Y[j]);                                                                                                                        
}

Here are the screenshots with the system working: