Logistic Regression doesnt learn anything

asked 2019-11-26 22:28:21 -0500

evgevd gravatar image

I train binary LR in python (opencv 3.1 and 4.1) and get always prediction = 1. When I try it on sklearn it works. OpenCV DT and ANN work too.

I tried to train LR with different parametres. Actual version:

dtype = np.float32
train_data = cv.ml.TrainData_create(train_features, cv.ml.ROW_SAMPLE, train_labels.astype(dtype))
cv_model = cv.ml.LogisticRegression_create()
cv_model.setLearningRate(5e-5)
cv_model.setRegularization(cv.ml.LogisticRegression_REG_L2)
cv_model.setTrainMethod(cv.ml.LogisticRegression_MINI_BATCH)
cv_model.setMiniBatchSize(128)
cv_model.setTermCriteria((cv.TERM_CRITERIA_COUNT, int(1e+7),  1e-7))
cv_model.train(train_data)
edit retag flag offensive close merge delete

Comments

Did you normalized your data ?

holger gravatar imageholger ( 2019-11-26 23:04:47 -0500 )edit

No I didnt. Is it necessary in this case?

evgevd gravatar imageevgevd ( 2019-11-26 23:07:05 -0500 )edit

what is train_data.size() ?

berak gravatar imageberak ( 2019-11-27 00:55:35 -0500 )edit
1

train_features.shape ((448, 4) train_labels.shape (448,))

evgevd gravatar imageevgevd ( 2019-11-27 01:02:02 -0500 )edit

So you have 4 features with 448 data rows in total? Can you also post the code where you do your prediction(where it goes wrong)? I am no expert - at least the training code looks good to me (i dont know the range of your feature values and if the need to be normalized at all - i could imagine sklearn does this for you - only a wild guess).

holger gravatar imageholger ( 2019-11-27 06:05:34 -0500 )edit

I tried with normalization and got the same result:

from sklearn.preprocessing import normalize
train_features = normalize(train_features, axis=0)

I make prediction by this:

cv_model.predict(train_features)
evgevd gravatar imageevgevd ( 2019-11-27 07:50:54 -0500 )edit
1

https://github.com/opencv/opencv/blob...

(i guess, your learningRate is too low)

berak gravatar imageberak ( 2019-11-27 08:19:20 -0500 )edit
1

Good point berak - But with a low value it just takes long to converge. How about picking another term criteria - instead of count use accuracy? I just checked my yolo config - it has learning rate of 0.0005

holger gravatar imageholger ( 2019-11-27 08:50:53 -0500 )edit