What is the difference between linear regression and logistic regression? [closed]

by

  • Linear regression output as probabilities

    It’s tempting to use the linear regression output as probabilities but it’s a mistake because the output can be negative, and greater than 1 whereas probability can not. As regression might actually
    produce probabilities that could be less than 0, or even bigger than
    1, logistic regression was introduced.

    Source: http://gerardnico.com/wiki/data_mining/simple_logistic_regression

    enter image description here

  • Outcome

    In linear regression, the outcome (dependent variable) is continuous.
    It can have any one of an infinite number of possible values.

    In logistic regression, the outcome (dependent variable) has only a limited number of possible values.

  • The dependent variable

    Logistic regression is used when the response variable is categorical in nature. For instance, yes/no, true/false, red/green/blue,
    1st/2nd/3rd/4th, etc.

    Linear regression is used when your response variable is continuous. For instance, weight, height, number of hours, etc.

  • Equation

    Linear regression gives an equation which is of the form Y = mX + C,
    means equation with degree 1.

    However, logistic regression gives an equation which is of the form
    Y = eX + e-X

  • Coefficient interpretation

    In linear regression, the coefficient interpretation of independent variables are quite straightforward (i.e. holding all other variables constant, with a unit increase in this variable, the dependent variable is expected to increase/decrease by xxx).

    However, in logistic regression, depends on the family (binomial, Poisson,
    etc.) and link (log, logit, inverse-log, etc.) you use, the interpretation is different.

  • Error minimization technique

    Linear regression uses ordinary least squares method to minimise the
    errors and arrive at a best possible fit, while logistic regression
    uses maximum likelihood method to arrive at the solution.

    Linear regression is usually solved by minimizing the least squares error of the model to the data, therefore large errors are penalized quadratically.

    Logistic regression is just the opposite. Using the logistic loss function causes large errors to be penalized to an asymptotically constant.

    Consider linear regression on categorical {0, 1} outcomes to see why this is a problem. If your model predicts the outcome is 38, when the truth is 1, you’ve lost nothing. Linear regression would try to reduce that 38, logistic wouldn’t (as much)2.

More Related Contents:

  1. Why does one hot encoding improve machine learning performance? [closed]
  2. Which algorithms involves in deep learning? [closed]
  3. Machine Learning Two class classification [closed]
  4. Artificial Neural Network Toplogy
  5. Whats the difference between a test set and a train set?
  6. How to interpret data output from Weka?
  7. What function defines accuracy in Keras when the loss is mean squared error (MSE)?
  8. What is the role of the bias in neural networks? [closed]
  9. Loss & accuracy – Are these reasonable learning curves?
  10. Clarification on a Neural Net that plays Snake
  11. Accuracy Score ValueError: Can’t Handle mix of binary and continuous target
  12. Many to one and many to many LSTM examples in Keras
  13. gradient descent using python and numpy
  14. How to interpret caffe log with debug_info?
  15. Logo recognition in images [closed]
  16. Caffe | solver.prototxt values setting strategy
  17. What is the difference between loss function and metric in Keras? [duplicate]
  18. ResNet: 100% accuracy during training, but 33% prediction accuracy with the same data
  19. Is deep learning bad at fitting simple non linear functions outside training scope (extrapolating)?
  20. How to interpret loss and accuracy for a machine learning model [closed]
  21. How to put more weight on certain features in machine learning?
  22. Can Keras deal with input images with different size?
  23. Keras: model.evaluate vs model.predict accuracy difference in multi-class NLP task
  24. How to create caffe.deploy from train.prototxt
  25. What is the difference between supervised learning and unsupervised learning? [closed]
  26. Which machine learning classifier to choose, in general? [closed]
  27. How can I get biases from a trained model in Keras?
  28. Machine learning – Linear regression using batch gradient descent
  29. Error in Confusion Matrix : the data and reference factors must have the same number of levels
  30. Under what parameters are SVC and LinearSVC in scikit-learn equivalent?

Leave a Comment