Data Science Stack Exchange Community Digest

Top new questions this week:

Remove or not to remove outliers

Are there any known academic sources that point towards supporting not removing outliers? Let say if the outlier is a natural occurrence or it has relationship to the value of target variable

outlier

asked by Kusisi Karem Score of 5 answered by Lynn Score of 8

What does it mean when a model is more conservative?

I’m trying to tune some parameters in XGBoost and read a lot about “…makes to model more conservative”. Can somebody explain me what the word conservative means in this case? I can imagine …

xgboost hyperparameter-tuning

asked by tturbo Score of 4 answered by German C M Score of 3

Why is Data with an Overrepresented Class called Imbalanced not Unbalanced?

I’ve seen the term Imbalanced used to described data that has an over-representation of one class. What’s the reasoning behind naming this type of data Imbalanced as opposed to Unbalanced, which seems …

class-imbalance terminology imbalanced-data

asked by Connor Score of 3 answered by the_strange Score of 3

Is applying dropout the same as zeroing random neurons?

Is applying dropout equivalent to zeroing output of random neurons in each mini-batch iteration and leaving rest of forward and backward steps in backpropagation unchanged? I’m implementing network …

deep-learning neural-network dropout

asked by Qbik Score of 3 answered by hH1sG0n3 Score of 5

What is the best way to determine if there is variable interactivity between independent parameters in a prediction model

OK, the best way to describe this is with an example. (admittedly simplified) I want to predict the speed of drivers on a motorway and I have two input variables the nationality of the driver how …

predictive-modeling linear-regression logistic-regression hyperparameter-tuning

asked by Jay Ramsay Score of 2 answered by Brian Spiering Score of 1

Understanding the relationship between two features

I am trying to solve classification task. Could you suggest me, if these two features are independent? the plot looks strange

classification pandas data-analysis feature-scaling matplotlib

asked by Любовь Пономарева Score of 1 answered by Brian Spiering Score of 2

Two computers, different outputs for same code in R studio

So a classmate and I are working with the same data and code for a problem in a data analysis class. My code for this section looks like this: …

r statistics rstudio non-parametric

asked by SpaceManSpiff Score of 1 answered by Erwan Score of 0

Greatest hits from previous weeks:

How do you visualize neural network architectures?

When writing a paper / making a presentation about a topic which is about neural networks, one usually visualizes the networks architecture. What are good / simple ways to visualize common …

machine-learning neural-network deep-learning visualization

asked by Martin Thoma Score of 157 answered by Alex Lenail Score of 76

Merging multiple data frames row-wise in PySpark

I have 10 data frames pyspark.sql.dataframe.DataFrame, obtained from randomSplit as …

python apache-spark cross-validation pyspark

asked by krishna Prasad Score of 34 answered by Jan van der Vegt Score of 57

Cross-entropy loss explanation

Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...

machine-learning neural-network deep-learning softmax

asked by enterML Score of 72 answered by Neil Slater Score of 96

What is the difference between Gradient Descent and Stochastic Gradient Descent?

What is the difference between Gradient Descent and Stochastic Gradient Descent? I am not very familiar with these, can you describe the difference with a short example?

machine-learning neural-network deep-learning gradient-descent

asked by Developer Score of 75 answered by Sociopath Score of 82

Calculation and Visualization of Correlation Matrix with Pandas

I have a pandas data frame with several entries, and I want to calculate the correlation between the income of some type of stores. There are a number of stores with income data, classification of …

python statistics visualization pandas

asked by gdlm Score of 38 answered by AN6U5 Score of 28

Difference between isna() and isnull() in pandas

I have been using pandas for quite some time. But, I don’t understand what’s the difference between isna() and isnull(). And, …

python pandas dataframe

asked by Vaibhav Thakur Score of 184 answered by Djib2011 Score of 229

How to get accuracy, F1, precision and recall, for a keras model?

I want to compute the precision, recall and F1-score for my binary KerasClassifier model, but don’t find any solution. Here’s my actual code: …

machine-learning neural-network deep-learning classification keras

asked by ZelelB Score of 56 answered by Tasos Score of 58

Can you answer these questions?

Understanding Syntactic divergence

I am trying to read a paper arxiv.org/abs/2004.14444 Section 6.1 of the paper describes Syntactic divergence. I have confusion regarding the distribution graphs and split of the dataset. If a …

machine-learning nlp dataset distribution stanford-nlp

asked by jon Score of 1

Can a child node have an information gain that is greater than the entropy of the parent?

I feel like the answer to this question is an obvious “no,” but I wanted to get an answer from people who definitely know more about this than I. Mostly asking because I’m curious about …

classification

asked by Samuel Emmett Score of 1

Data science for brain signal processing

I want to create a model which analyzes brain signals and detect if an action is performed (e.g., move hand, move leg). I’ve seen several studies on the subject, however I’m not sure if the topic is …

machine-learning classification data-science-model

asked by Carine Score of 1