Data Science Stack Exchange Community Digest

Top new questions this week:

Is it Possible to plot Scatter Plot + Histogram + Correlation Values in a single plot (in python)?

I recently came across corrmorant package in R. It allows to plot all three basic EDA plots together: Scatter Plot + Histogram + Correlation Values. Is it possible …

python data-analysis

asked by pqrz Score of 2 answered by Nicolas Martin Score of 4

What is the difference (interpretation) between the partial R^2 and the SHAP value for a linear regression model?

To calculate the coefficient of partial determination R2 for a given variable: We calculate the R2 with and without that variable and substract them. This implies fitting a different model with and …

correlation explainable-ai shap

asked by skan Score of 2 answered by skan Score of 1

How to remove test set so that model uses all data as training data?

I have developed a RandomForest classification model and I am pretty satisfied with the results on the test set. Now, my next step is to deploy the model. Before …

machine-learning python classification scikit-learn random-forest

asked by Nemo_the_scientist Score of 1 answered by Quan Nguyen Score of 1

Test-set error in cross-validation with regularization

I am performing regularization (Ridge regression) using cross validation. I understand that for a certain value of the regularization parameter $\lambda$, we first fit on the training-set and then we …

cross-validation regularization

asked by dimitris Score of 1 answered by Brian Spiering Score of 0

How to split train/test datasets according to labels' classes

I faced a problem while I using sklearn.train_test_split(). Here is the code I use. …

machine-learning scikit-learn multiclass-classification

asked by yuyang sun Score of 1 answered by db_max Score of 1

Should I remove objects labeled “unknown” from my test set?

I’m using an object detector (yolo v5) to detect and identify fish in underwater video. I have a labeled data set, but some fish are difficult for the annotator to correctly identify, so they get …

yolo

asked by Ketil Malde Score of 1 answered by Erwan Score of 2

How to find entity names in non-grammaratical text?

Given: Bella Pharma Rosuvas 5 Enalapril 10 Domperidone 10 Ned’s 24 by 7 PCM 650 Teneligliptin 5 … Get: …

classification nlp few-shot-learning

asked by Ryu J. Score of 1 answered by Erwan Score of 0

Greatest hits from previous weeks:

Difference between isna() and isnull() in pandas

I have been using pandas for quite some time. But, I don’t understand what’s the difference between isna() and isnull(). And, …

python pandas dataframe

asked by Vaibhav Thakur Score of 185 answered by Djib2011 Score of 230

The cross-entropy error function in neural networks

In the MNIST For ML Beginners they define cross-entropy as $$H_{y’} (y) := – \sum_{i} y_{i}’ \log (y_i)$$ $y_i$ is the predicted probability value for class $i$ and $y_i’$ is the true probability …

machine-learning tensorflow

asked by Martin Thoma Score of 149 answered by KT. Score of 125

What is weight and bias in deep learning?

I’m starting to learn Machine learning from Tensorflow website. I have developed a very very rudimentary understanding of the flow a deep learning program follows (this method makes me learn fast …

machine-learning deep-learning tensorflow

asked by Umer Farooq Score of 28 answered by Declan Score of 22

Remove part of string in R

I have a table in R. It just has two columns and many rows. Each element is a string that contains some characters and some numbers. I need number part of the element. How can I have number part? For …

r dataset bioinformatics

asked by mahtab Score of 3 answered by Marmite Bomber Score of 7

When is precision more important over recall?

Can anyone give me some examples where precision is important and some examples where recall is important?

machine-learning evaluation

asked by Rajat Score of 43 answered by SmallChess Score of 63

When to use One Hot Encoding vs LabelEncoder vs DictVectorizor?

I have been building models with categorical data for a while now and when in this situation I basically default to using scikit-learn’s LabelEncoder function to transform this data prior to building …

scikit-learn categorical-data feature-engineering

asked by anthr Score of 169 answered by AN6U5 Score of 205

Sparse_categorical_crossentropy vs categorical_crossentropy (keras, accuracy)

Which is better for accuracy or are they the same? Of course, if you use categorical_crossentropy you use one hot encoding, and if you use …

neural-network keras loss-function encoding

asked by Master M Score of 66 answered by featuredpeow Score of 64

Can you answer these questions?

Data Leakage (specific case with standardization)

I read a few blog posts and found some books that caution against data leakage when adjusting the training data using any information from the held out datasets. For example, if I were to standardize …

data data-cleaning data-leakage simulation

asked by dzheng1887 Score of 1

N-ary decision tree with categorical features

I want to build an n-ary decision tree with categorical features. I am using ordinary ID3 algorithm to build a tree. Lets take the next dataset as a training dataset for building a decision tree: …

classification decision-trees categorical-data categorical-encoding cart

asked by dzi Score of 1 answered by Erwan Score of 0

A mathematician from the outside looking in

I am wondering if anybody could give a survey of applications of approximation theory to data science. One application I am familiar with are, for example, wavelet neural networks. Does anybody know …

neural-network mathematics kernel

asked by Joe Shmo Score of 1