Data Science Stack Exchange Community Digest

Top new questions this week:

How can someone build a dataset for a “propensity to purchase” model?

Ok, this might seem a trivial question for some and it’s not even a question, more like a discussion. I read the rules and I believe it’s everything fine, so I’m gonna take my chances… Here’s the …

machine-learning predictive-modeling training preprocessing

asked by Andrew Joplh Score of 3

Is the hashing-trick just a random clustering algorithm when used on e.g user-id?

Say I have 1.000.000 user-ids and I choose to use the (signed) hashing-trick with a hash-vector length of 500.000. Wouldn’t that effectively just mean that half of the time, we would have two …

hashing-trick

asked by CutePoison Score of 2 answered by Ben Reiniger Score of 1

Multiple classes present in one-hot encoding

When dealing with classification for multiple classes present in the same sample, can the output layer have the form of one-hot encoding, but instead of only one hot, have multiple? That is, in case …

neural-network multiclass-classification multilabel-classification one-hot-encoding

asked by smone Score of 2 answered by Broele Score of 4

Neural network not learning at all

I am training a MLP on a tabular dataset, the pendigits dataset. Problem is that training loss and accuracy are more or less stable, while validation and test loss and accuracy are completely constant….

pytorch loss-function accuracy mlp csv

asked by CasellaJr Score of 1 answered by Iya Lee Score of 0

In Q-learning, why does Q index on both state and action?

In Q-learning, Q is an array of expected rewards for (state, action) combinations. It seems to me the same result could be achieved while slightly simplifying the algorithm, if instead of associating …

reinforcement-learning q-learning

asked by rwallace Score of 1 answered by mohottnad Score of 1

Using BERT to extract a list of words and phrases from documents

I have a list of words and phrases (~3k items). What are my options to extract them from documents (~3M of job descriptions) with NLP? I do not have labeled data. For example my list of words and …

nlp transformer bert information-extraction

asked by E.K. Score of 1 answered by Erwan Score of 0

How to bias a neural network towards one category in binary classification?

I have a basic sequential neural network built with TensorFlow. …

neural-network tensorflow binary-classification

asked by Chandler Kenworthy Score of 1 answered by dx2-66 Score of 1

Greatest hits from previous weeks:

How to set batch_size, steps_per epoch, and validation steps?

I am starting to learn CNNs using Keras. I am using the theano backend. I don’t understand how to set values to: batch_size …

machine-learning keras cnn theano

asked by Ermene Score of 59 answered by Silpion Score of 57

Calculation and Visualization of Correlation Matrix with Pandas

I have a pandas data frame with several entries, and I want to calculate the correlation between the income of some type of stores. There are a number of stores with income data, classification of …

python statistics visualization pandas

asked by gdlm Score of 38 answered by AN6U5 Score of 28

How to plot two columns of single DataFrame on Y axis

I have two data frames (Action, Comedy). Action contains two columns (year, rating) ratings columns contains average rating with respect to year. The Comedy data frame contains the same two columns …

python pandas visualization dataframe

asked by Bilal Butt Score of 18 answered by user2333453 Score of 20

ValueError: Input contains NaN, infinity or a value too large for dtype('float32')

I got ValueError when predicting test data using a RandomForest model. My code: …

python scikit-learn pandas random-forest python-3.x

asked by Edamame Score of 94 answered by fernando Score of 76

Perform k-means clustering over multiple columns

I am trying to perform k-means clustering on multiple columns. My data set is composed of 4 numerical columns and 1 categorical column. I already researched previous questions but the answers are not …

python clustering pandas

asked by Lola Score of 13 answered by Simon Larsson Score of 19

How to get correlation between two categorical variable and a categorical variable and continuous variable?

I am building a regression model and I need to calculate the below to check for correlations Correlation between 2 Multi level categorical variables Correlation between a Multi level categorical …

r statistics correlation

asked by GeorgeOfTheRF Score of 131 answered by Alexey Grigorev Score of 123

What's the difference between fit and fit_transform in scikit-learn models?

I do not understand the difference between the fit and fit_transform methods in scikit-learn. Can anybody explain simply why we …

python scikit-learn

asked by Kaggle Score of 238 answered by K3—rnc Score of 241

Can you answer these questions?

Best ML models for long term time series forecast

I have a project to make a long term prediction (like 5 years) of electricity production by types of power plants (solor, wind, coal, nuclear etc.). I have access to time series data in MW [megawatts] …

machine-learning time-series forecasting forecast

asked by Dan Jírovec Score of 1 answered by Iya Lee Score of 0

Does high number of output labels affect the performance of BERT and how to handle the class imbalance issue while doing multi text classification?

I am using BERT to do multiclass text classification. The number of output classes I have to predict from is: 116 and there is high degree of class imbalance that I see. We have the following kind of …

machine-learning nlp transformer bert text-classification

asked by Swagat Mishra Score of 1 answered by Erwan Score of 0

Why is my predictor value (continuous) perfectly correlated with my logit value (when testing logistic regression model assumptions)?

Question: Why is my predictor value (continuous) perfectly correlated with my logit value (when testing logistic regression model assumptions)? Code: …

regression logistic-regression logistic

asked by In_cognito Score of 1 answered by Dave Score of 0